Search papers, labs, and topics across Lattice.
The authors introduce ManiTwin, an automated pipeline for converting single images into simulation-ready, semantically annotated 3D assets for robotic manipulation. This pipeline efficiently generates high-quality 3D assets with physical properties, language descriptions, functional annotations, and verified manipulation proposals. Using this pipeline, they construct ManiTwin-100K, a dataset containing 100K diverse and annotated 3D assets, demonstrating its utility for manipulation data generation, scene synthesis, and VQA.
Forget painstakingly creating 3D assets for robot training - ManiTwin automates the process, turning single images into simulation-ready objects at scale.
Learning in simulation provides a useful foundation for scaling robotic manipulation capabilities. However, this paradigm often suffers from a lack of data-generation-ready digital assets, in both scale and diversity. In this work, we present ManiTwin, an automated and efficient pipeline for generating data-generation-ready digital object twins. Our pipeline transforms a single image into simulation-ready and semantically annotated 3D asset, enabling large-scale robotic manipulation data generation. Using this pipeline, we construct ManiTwin-100K, a dataset containing 100K high-quality annotated 3D assets. Each asset is equipped with physical properties, language descriptions, functional annotations, and verified manipulation proposals. Experiments demonstrate that ManiTwin provides an efficient asset synthesis and annotation workflow, and that ManiTwin-100K offers high-quality and diverse assets for manipulation data generation, random scene synthesis, and VQA data generation, establishing a strong foundation for scalable simulation data synthesis and policy learning. Our webpage is available at https://manitwin.github.io/.
Tsinghua AI