Search papers, labs, and topics across Lattice.
The authors introduce PointQ-Bench, a new benchmark for point cloud quality assessment (PCQA) that moves beyond scalar scoring to focus on diagnostic and interpretable quality understanding. The benchmark includes 3,083 point clouds with diverse issues, annotated with MOS, quality levels, issue tags, descriptions, and QA pairs, supporting tasks like anomaly sensing, defect diagnosis, usability grading, and open-ended quality reporting. Experiments on 14 vision-language models and PCQA baselines reveal a perception-diagnosis gap, where models struggle with grounded diagnosis and quality calibration despite emerging defect perception abilities.
Current vision-language models can *see* point cloud defects, but can't reliably *diagnose* them, highlighting a critical gap in grounded quality understanding.
Point cloud quality plays a critical role in 3D acquisition, reconstruction, rendering, and perception, yet existing point cloud quality assessment (PCQA) research remains largely centered on scalar score prediction. In practical inspection scenarios, quality assessment often involves identifying defects, characterizing dominant issue types, assessing downstream usability, and providing evidence-supported descriptions, which are not explicitly evaluated by current benchmarks. We introduce PointQ-Bench, a benchmark designed to extend PCQA from scalar scoring toward comprehensive quality understanding. PointQ-Bench consists of 3,083 point clouds spanning authentic scans, simulated distortions, and AI-generated content, covering eight major issue types. Each sample is annotated with mean opinion scores (MOS), quality levels, issue tags, expert-grounded descriptions, and 12,332 question-answer pairs. The benchmark supports three perception-oriented tasks: anomaly sensing, defect diagnosis, and usability grading, as well as a cognition-oriented task of open-ended quality reporting. To evaluate free-form quality descriptions, we further propose SSFRQ-5D, a five-dimensional evaluation protocol validated through human-AI agreement analysis. Extensive experiments on 14 vision-language models and traditional PCQA baselines reveal a consistent perception-diagnosis gap: while current models exhibit emerging abilities in coarse defect perception, they struggle with grounded diagnosis and quality calibration. Strong 2D MLLMs generally outperform existing 3D VLMs, and the benefit of additional views or point-level inputs is non-uniform, varying across tasks, data sources, and models, particularly under boundary-ambiguous conditions. Overall, PointQ-Bench provides a diagnostic testbed for advancing reliable and interpretable point cloud quality understanding.
