Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an asynchronous architecture for 3D scene graph construction that allows for concurrent lightweight online mapping and heavyweight semantic refinement, addressing the limitations of traditional sequential methods. By employing a probabilistic voxel-based backbone, the system maintains stable object identities while leveraging background vision-language models to enrich the graph dynamically. The proposed approach not only matches but also outperforms existing methods on semantic segmentation and visual grounding benchmarks, achieving significant improvements in accuracy.
Asynchronous mapping and semantic refinement enable real-time querying of 3D scene graphs, enhancing both exploration and understanding of complex environments.
Open-vocabulary 3D scene graph methods typically operate in two stages: first reconstruct, then enrich with vision-language models, leaving the graph unqueryable during exploration. We argue that this sequential coupling is unnecessary and propose an asynchronous architecture in which lightweight online mapping runs concurrently with heavyweight semantic refinement. A probabilistic voxel-based backbone maintains stable object identities incrementally, while background VLM agents progressively enrich the graph. This framework resolves duplicate object tracks through semantic loop closure, attaches fine-grained visual attributes and derives spatial relations between objects. A multi-target frame scheduler amortizes VLM cost by selecting a small set of informative frames that jointly cover multiple targets. The resulting scene graph is queryable during exploration and grows in semantic richness over time. Our method matches or outperforms existing open-vocabulary 3D scene graph methods on semantic segmentation (ScanNet, Replica) and surpasses the prior state-of-the-art across three visual grounding benchmarks (Sr3D+, Nr3D, ScanRefer) by 15.3 to 18.8 A@0.25. Project page: https://denizbickici.github.io/thinkgraphs/