Search papers, labs, and topics across Lattice.
This paper addresses the challenge of glass surface detection (GSD) by shifting the focus from traditional 2D appearance cues to a 3D visual geometry framework. By leveraging a visual geometry grounded transformer (VGGT) to distill 3D priors and employing a multi-task learning approach with a Frequency Self-Attention Module (FSAM) and a Geometry Grounding Block (GeGB), the authors achieve significant improvements in glass surface localization and segmentation. The proposed method outperforms existing techniques across seven benchmarks and enhances scene reconstruction in complex glass environments.
Grounding glass surface detection in 3D geometry leads to state-of-the-art performance and improved scene reconstruction, challenging the limitations of traditional 2D methods.
Glass surface detection (GSD) is critical for scene understanding and reconstruction, and yet remains challenging due to the transparency and reflectivity of glass surfaces. Existing GSD methods typically rely on 2D appearance cues, which may fail in geometrically ambiguous scenes. In this paper, we propose a paradigm shift: grounding GSD in 3D visual geometry to explicitly model the physical existence of glass surfaces. Our method first distills rich 3D priors from the visual geometry grounded transformer (VGGT) and generates glass-aware 3D representations. It then exploits multi-tasking learning with a novel glass detection head, consisting of two core modules: a Frequency Self-Attention Module (FSAM) that identifies glass-specific spectral features for glass surface localization, and a Geometry Grounding Block (GeGB) that selectively grounds 2D features in 3D geometry for glass surface segmentation. Extensive experiments demonstrate that our method achieves state-of-the-art performance across seven standard GSD benchmarks, generalizes well to video/multi-modal data, and substantially improves reconstruction in glass scenes. Code is available in https://github.com/YT3DVision/VGGT_GLASS.
