Search papers, labs, and topics across Lattice.
This paper introduces SILICA, a novel pipeline that utilizes diffusion model priors to achieve joint glass segmentation and depth estimation, addressing the limitations of standard depth sensors on transparent surfaces. By leveraging mutual information exchange between segmentation and depth prediction, SILICA eliminates the need for paired real-world glass depth annotations, enabling robust performance across diverse environments. Experimental results show that SILICA outperforms existing models by nearly 20%, establishing a new benchmark in transparent surface perception and enhancing navigation safety in 3D mapping applications.
SILICA achieves a nearly 20% improvement in depth estimation accuracy for transparent surfaces without requiring real-world glass depth annotations.
Standard depth sensors systematically fail on transparent surfaces, creating corrupted 3D maps and severe navigation hazards. While specialized hardware sensors can detect glass, they lack modularity and have extensive hardware dependencies. Consequently, learning-based monocular depth estimation has emerged as a compelling alternative. However, domain-specific glass-aware monocular depth estimators struggle with unfamiliar indoor layouts; restricted by the severe scarcity of real-world glass depth annotations, they fail to generalize zero-shot to new settings. This motivates us to explore whether the extensive priors of text-to-image diffusion models can enable generalizable perception of transparent surfaces. We introduce SILICA, a unified pipeline leveraging these priors to jointly predict glass segmentation and glass-aware depth. This mutual information exchange establishes a robust spatial hierarchy, entirely eliminating the need for paired real-world glass depth annotations. Subsequently, we use the predicted segmentation mask to explicitly filter incorrect glass depth points from standard sensors, recovering accurate metric glass depth for downstream 3D mapping and autonomous collision avoidance. Supported by our novel Mirage 18k dataset, extensive experiments demonstrate that SILICA achieves remarkable zero-shot transfer across diverse, unseen environments, outperforming state-of-the-art models by almost 20% and setting a new benchmark for transparent surface perception.
