Search papers, labs, and topics across Lattice.
This paper introduces OP3DSG, a novel framework for generating open-vocabulary part-aware 3D scene graphs (3DSGs) that enhances the representation of real-world environments by integrating detailed object-part knowledge and spatial relations. The approach addresses limitations of existing object-centric models by preserving small, interactive components and employing a geometry-initialized prior graph refined by large language models to minimize erroneous relational predictions. Experimental results on the newly proposed UniGraph3D benchmark reveal that OP3DSG outperforms previous methods, establishing itself as a robust perception backbone for various robotics applications.
OP3DSG achieves state-of-the-art performance in 3D scene graph generation by capturing intricate spatial and functional relationships in real-world environments.
3D scene graphs (3DSGs) provide a compact and structured abstraction of 3D environments. Although advances in foundation models have enabled open-vocabulary 3DSG generation, existing approaches remain object-centric and encode limited relational information -- restricting their applicability in real-world scenarios that require fine-grained understanding. We propose OP3DSG, an open-vocabulary part-aware 3DSG generation framework that constructs unified graphs that jointly model objects, interactive parts, spatial relations, functional relations, and affordances. OP3DSG integrates object-part knowledge-guided detection with part-aware 3D fusion to preserve small and interaction-relevant components, and employs a geometry-initialized prior graph with LLM-based refinement to reduce spurious relational predictions while enabling efficient graph construction. To systematically evaluate unified 3D scene graph construction, we introduce UniGraph3D, a benchmark designed for part-aware perception and multi-level relational reasoning. Experimental results show that OP3DSG achieves state-of-the-art performance and demonstrates its effectiveness as a perception backbone in diverse real-world robotics tasks.
