Search papers, labs, and topics across Lattice.
This paper introduces a multimodal quality assessment framework for 3D Gaussian Splatting (3DGS) that addresses the limitations of traditional image quality assessment methods by incorporating scene-level factors such as spatial structure and cross-view consistency. By leveraging a VGGT-based encoder augmented with a quality head, the framework learns structure-aware quality representations that integrate depth and point-cloud information, enhancing the reliability of quality evaluations. The approach culminates in a grounded multimodal reasoning mechanism utilizing a Qwen-based MLLM to assess the quality of 3DGS scenes more effectively than existing methods.
Traditional image quality assessment falls short for 3D scenes, but a new framework captures spatial structure and consistency, revolutionizing how we evaluate 3D Gaussian Splatting.
3D Gaussian Splatting (3DGS) has emerged as an effective representation for novel view synthesis and 3D scene reconstruction, creating an increasing demand for reliable quality assessment. Unlike conventional image quality assessment (IQA), the quality of a 3DGS scene depends not only on the perceptual fidelity of rendered views, but also on scene-level factors such as spatial structure and cross-view consistency. Existing IQA methods are limited by their reliance on 2D perceptual cues, whereas general multimodal large language models (MLLMs) are not designed for stable quality regression and may produce unreliable judgments. To address these limitations, a multimodal quality assessment framework is developed for 3DGS scene understanding. First, a 3D-aware quality representation learning framework is introduced by augmenting a VGGT-based encoder with a dedicated quality head. Multi-view images are encoded into view-specific features and aggregated to capture cross-view consistency, while geometric cues are incorporated through joint modeling of depth and point-cloud-related structural information, enabling the learning of structure-aware quality representations beyond appearance-driven features. Second, a grounded multimodal reasoning mechanism is constructed by jointly feeding original images, depth maps, point cloud renderings, and camera parameters into a Qwen-based MLLM.
