Search papers, labs, and topics across Lattice.
This paper introduces Prior-SG, a novel framework for generating hierarchical 3D scene graphs that addresses the limitations of existing methods in arbitrarily-structured environments. By framing scene graph generation as a probabilistic alignment problem and leveraging a multi-scale, open-vocabulary feature fusion strategy, Prior-SG effectively integrates RGB-D sensor data to create a grounded Instance Graph. The approach not only achieves state-of-the-art semantic region segmentation but also allows for dynamic restructuring of spatial partitions based on high-level tasks, showcasing significant advancements in handling complex environments without strict geometric constraints.
Prior-SG enables robots to redefine their spatial understanding in real-time, achieving zero-shot flexibility in semantic region segmentation even in the absence of physical boundaries.
Hierarchical 3D scene graphs are a promising representation for high-level spatial reasoning in autonomous mobile platforms. However, existing extraction frameworks typically rely on purely local visual clustering or strict geometric heuristics, such as wall-separated rooms, which fail in open-plan or arbitrarily-structured environments. We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation fundamentally as a probabilistic alignment problem. As the robot explores, it continuously aggregates an incoming RGB-D sensor stream into a physically grounded Instance Graph utilizing a multi-scale, open-vocabulary feature fusion strategy. The system then infers the high-level functional semantics of this map through a Maximum A Posteriori (MAP) estimate, guided by a Prior Graph-a logical expectation of the environment's structure and task-relevant vocabulary synthesized dynamically by a Large Language Model. By optimizing a Markov Random Field that fuses heterogeneous experts (visual, geometric, and discrete objects) with these topological priors, the system resolves local perceptual ambiguities. We validate this approach across diverse simulated residential datasets and large, open-plan real-world environments. Prior-SG achieves state-of-the-art semantic region segmentation accuracy compared to recent baselines, robustly delineates distant functional boundaries in the absence of physical walls, and uniquely provides zero-shot ontological flexibility, enabling the robot to entirely restructure its spatial partitioning based on a given high-level task.
