Search papers, labs, and topics across Lattice.
This paper introduces PRISM, an innovative pipeline that generates personalized robotic datasets using a single image and natural-language instructions, addressing the challenges of limited generalization in robot policy learning. By creating semantically and geometrically aligned digital cousin scenes, PRISM enables the synthesis of executable demonstrations without the need for costly teleoperation. Experimental results demonstrate that policies trained on datasets generated by PRISM significantly outperform those trained on traditional methods, achieving up to a 100% success rate in real-world manipulation tasks and showing robustness across varied environments.
Policies trained on PRISM-generated datasets achieve unprecedented success rates in real-world tasks, outperforming traditional dataset methods by a wide margin.
Recent advances in large-scale pretrained vision-language-action models have improved robot policy learning, but directly deploying such policies in user-specific environments remains challenging due to limited generalization, which inevitably requires collecting a dataset tailored to the target environment. Teleoperation yields well-aligned data but is costly and difficult to scale, whereas simulation scales easily but struggles to resemble the target environment and generate task-specific trajectories. To meet both simultaneously, we propose PRISM, an end-to-end pipeline that generates personalized robotic datasets from a single image and a natural-language instruction. PRISM constructs digital cousin scenes that are semantically and geometrically aligned with the user environment yet diverse at the instance level, and synthesizes executable demonstrations without human teleoperation. Extensive experiments show that policies trained on PRISM-generated datasets outperform those trained on baseline-generated datasets on LIBERO and LIBERO-Plus, achieve up to 100\% success rate on three real-world manipulation tasks, and maintain stronger performance when evaluated in environments that differ from those seen during training.
