Search papers, labs, and topics across Lattice.
This paper introduces MonoVoc, a novel pipeline for open vocabulary 3D scene understanding that decouples geometric reconstruction from semantic integration, addressing limitations of existing methods that rely on multiview capture and expensive optimizations. By utilizing a standard monocular video sequence, MonoVoc generates a compact and interpretable object-level semantic Gaussian map without the need for heavy language embeddings during the mapping process. Evaluations on the Replica dataset show that this approach not only maintains strong rendering fidelity and competitive segmentation accuracy but also achieves a significant reduction in memory usage, making it a scalable solution for 3D retrieval and question answering.
Achieving an order-of-magnitude reduction in memory usage, MonoVoc enables efficient open-vocabulary 3D scene understanding directly from monocular video.
Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural language. However, current 3D Gaussian frameworks are often bottlenecked by restrictive multiview capture requirements, costly scene-specific optimization, and the massive memory overhead of storing dense language features. We present a novel, training-free pipeline that fundamentally reimagines this paradigm by explicitly decoupling 3D geometric reconstruction from semantic integration. Given a standard monocular video sequence as input, our method efficiently outputs a compact, highly interpretable, and fully searchable object-level semantic Gaussian map. Rather than entangling heavy language embeddings within the mapping loop, we extract geometry independently and ground semantics through a lightweight, modular post-processing framework. Extensive evaluations on the Replica dataset demonstrate that this decoupled architecture preserves strong rendering fidelity and competitive segmentation accuracy. Crucially, by replacing dense per-Gaussian storage with modular, object-level semantic embeddings, our approach delivers an order-of-magnitude reduction in memory usage compared to SOTA baselines. This provides a highly efficient, scalable, and practical solution for open-vocabulary 3D retrieval and question answering directly from everyday monocular video.
