Search papers, labs, and topics across Lattice.
This paper introduces Robotic Scene Cloning (RSC), a method for adapting pre-trained robot manipulation policies to new scenes without additional training data. RSC uses visual prompting to guide the generation of scene-consistent robot operation trajectories, enabling the transfer of textures and moderate shape adaptations. Experiments in simulation and the real world show that RSC improves policy generalization in target environments.
Bypass the need for extensive on-site data collection when deploying pre-trained robot models by visually prompting them to adapt to new scenes.
Modern robots can perform a wide range of simple tasks and adapt to diverse scenarios in the well-trained environment. However, deploying pre-trained robot models in real-world user scenarios remains challenging due to their limited zero-shot capabilities, often necessitating extensive on-site data collection. To address this issue, we propose Robotic Scene Cloning (RSC), a novel method designed for scene-specific adaptation by editing existing robot operation trajectories. RSC achieves accurate and scene-consistent sample generation by leveraging a visual prompting mechanism and a carefully tuned condition injection module. Not only transferring textures but also performing moderate shape adaptations in response to the visual prompts, RSC demonstrates reliable task performance across a variety of object types. Experiments across various simulated and real-world environments demonstrate that RSC significantly enhances policy generalization in target environments.
