Search papers, labs, and topics across Lattice.
PhysGraph is a novel framework that integrates symbolic reasoning with structured 3D geometry to effectively model kinematic and physical properties in complex scenes. By utilizing RGB-D observations, it reconstructs object-centric 3D geometries, decomposes objects into functional parts, and infers materials and articulations, addressing the limitations of existing methods that focus primarily on semantic retrieval. Evaluated on both synthetic and real-world datasets, PhysGraph achieves state-of-the-art performance in tasks such as semantic segmentation, multi-object mass estimation, and articulation prediction, showcasing its potential for enhancing robotic perception and reasoning capabilities.
PhysGraph achieves state-of-the-art results in multi-object mass estimation and articulation prediction by seamlessly integrating physical properties into 3D scene representations.
To perform a wide range of daily tasks, robots need to construct a 3D representation that is semantically rich, physically grounded, and structured enough to support task planning and affordance prediction. However, existing approaches primarily focus on semantic retrieval, often overlooking physical and kinematic factors. Methods that attempt to model physical properties typically rely on narrow training sets or single-object modeling, limiting scalability and generalization across diverse object types. To address these challenges, we present PhysGraph, a framework that unifies symbolic reasoning with structured 3D geometry to model kinematic and physical properties in cluttered scenes. Given RGB-D observations, PhysGraph reconstructs object-centric 3D geometry and associates object instances across views. It then decomposes objects into functional parts and infers materials and articulations through visual reasoning. Evaluated on both synthetic and real-world datasets, PhysGraph achieves state-of-the-art results in semantic segmentation, multi-object mass estimation, and articulation prediction. With its simple yet effective design, PhysGraph produces physically consistent and semantically structured scene graphs, serving as a structured 3D representation for downstream tasks such as constraint-aware 3D affordance prediction and real-to-sim transfer, both of which are demonstrated in our experiments.
