Search papers, labs, and topics across Lattice.
RecGen, a generative framework, is introduced for probabilistic joint estimation of object and part shapes, along with their pose, from sparse RGB-D observations. It leverages compositional synthetic scene generation and strong 3D shape priors to generalize across diverse object types and real-world environments. The method achieves state-of-the-art performance on complex, heavily occluded datasets, outperforming SAM3D by significant margins in geometric shape quality, texture reconstruction, and pose estimation while using fewer training meshes.
Achieve state-of-the-art 3D scene reconstruction from sparse views with 80% less training data by learning to generate, not just match, 3D structures.
Accurately reconstructing complex full multi-object scenes from sparse observations remains a core challenge in computer vision and a key step toward scalable and reliable simulation for robotics. In this work, we introduce RecGen, a generative framework for probabilistic joint estimation of object and part shapes, as well as their pose under occlusion and partial visibility from one or multiple RGB-D images. By leveraging compositional synthetic scene generation and strong 3D shape priors, RecGen generalizes across diverse object types and real-world environments. RecGen achieves state-of-the-art performance on complex, heavily occluded datasets, robustly handling severe occlusions, symmetric objects, object parts, and intricate geometry and texture. Despite using nearly 80% fewer training meshes than the previous state of the art SAM3D, RecGen outperforms it by 30.1% in geometric shape quality, 9.1% in texture reconstruction, and 33.9% in pose estimation.
