Search papers, labs, and topics across Lattice.
This paper addresses the problem of open-vocabulary 3D occupancy prediction in indoor scenes, which is crucial for embodied agents navigating complex environments. They introduce a framework based on 3D Language-Embedded Gaussians and geometry-only supervision to overcome the limitations of existing methods when applied to indoor environments. The key result is a significant improvement in open-vocabulary occupancy prediction on the Occ-ScanNet dataset, achieving 59.50 IoU and 21.05 mIoU.
Achieve state-of-the-art open-vocabulary occupancy prediction in complex indoor scenes by using only binary occupancy labels and a novel Gaussian-based representation.
Open-vocabulary 3D occupancy is vital for embodied agents, which need to understand complex indoor environments where semantic categories are abundant and evolve beyond fixed taxonomies. While recent work has explored open-vocabulary occupancy in outdoor driving scenarios, such methods transfer poorly indoors, where geometry is denser, layouts are more intricate, and semantics are far more fine-grained. To address these challenges, we adopt a geometry-only supervision paradigm that uses only binary occupancy labels (occupied vs free). Our framework builds upon 3D Language-Embedded Gaussians, which serve as a unified intermediate representation coupling fine-grained 3D geometry with a language-aligned semantic embedding. On the geometry side, we find that existing Gaussian-to-Occupancy operators fail to converge under such weak supervision, and we introduce an opacity-aware, Poisson-based approach that stabilizes volumetric aggregation. On the semantic side, direct alignment between rendered features and open-vocabulary segmentation features suffers from feature mixing; we therefore propose a Progressive Temperature Decay schedule that gradually sharpens opacities during splatting, strengthening Gaussian-language alignment. On Occ-ScanNet, our framework achieves 59.50 IoU and 21.05 mIoU in the open-vocabulary setting, surpassing all existing occupancy methods in IoU and outperforming prior open-vocabulary approaches by a large margin in mIoU. Code will be released at https://github.com/JuIvyy/LegoOcc.
