Search papers, labs, and topics across Lattice.
This paper introduces UTTO, an uncertainty-guided test-time optimization framework designed for depth-only open-vocabulary 3D semantic segmentation, addressing the challenge of privacy-preserving perception in indoor environments. By leveraging uncertainty as a guidance signal, UTTO identifies unreliable semantic predictions and refines them using semantic priors from foundation models, thus enhancing the reliability of depth-only segmentation. Experimental results on multiple datasets, including ScanNet20, ScanNet40, and ScanNet200, show that UTTO significantly outperforms existing methods, demonstrating its effectiveness in maintaining privacy while improving segmentation accuracy.
Uncertainty can be harnessed to enhance the reliability of depth-only 3D semantic segmentation without compromising privacy.
Privacy-preserving perception is a critical requirement for deploying 3D scene understanding systems in real-world indoor environments, yet it remains underexplored in open-vocabulary 3D semantic segmentation. Existing methods typically rely on obtaining rich semantic cues from RGB images, which may expose privacy-sensitive visual information. Depth-only 3D geometry provides a privacy-preserving alternative, but the absence of appearance-based semantic cues makes open-vocabulary predictions highly uncertain and less reliable. Under this setting, we propose to convert uncertainty into a guidance signal to identify unreliable semantic responses and use semantic priors from foundation models to regularize their refinement. We present UTTO, an uncertainty-guided test-time optimization framework for depth-only open-vocabulary 3D semantic segmentation. Without additional training, experiments on ScanNet20, ScanNet40, and ScanNet200 demonstrate that UTTO consistently improves depth-only open-vocabulary 3D segmentation and outperforms representative baselines under privacy-preserving conditions.
