Search papers, labs, and topics across Lattice.
This paper introduces Cloak, a novel training approach that enables a Vision-Language-Action (VLA) model to achieve zero-shot cross-embodiment manipulation by masking the end-effector in the wrist camera's view. By leveraging real-time masking based on the robot's geometry, Cloak allows the model to perform embodiment-agnostic visual reasoning and generalize to previously unseen embodiments without requiring additional data collection. The results show that Cloak-VLA can successfully transfer its learned capabilities to various new embodiments while maintaining performance levels from the original training setup.
Cloak enables VLA models to seamlessly adapt to new robotic embodiments without any additional training data, revolutionizing the way we think about robotic adaptability.
We present Cloak, a training recipe that endows a Vision-Language-Action (VLA) model with zero-shot cross-embodiment transfer by cloaking the end-effector from its own wrist camera. The end-effector occupies a large and consistent region of the wrist view and masking it allows for embodiment-agnostic visual reasoning. Cloak renders a mask in simulation from the robot's known geometry, accurately and in real time, with no segmentation or generative models. During training, we augment the mask so the model generalizes to embodiments unseen at training time. We demonstrate the recipe with Cloak-VLA, a VLA trained with Cloak on a single parallel-jaw gripper dataset. No data of new embodiments is ever collected. Cloak-VLA transfers zero-shot to various unseen embodiments, including another gripper, another arm, and a five-fingered hand, while preserving the source embodiment's performance. By decoupling the wrist view from its own embodiment, Cloak allows data to outlive the hardware it was collected on.
Stanford HAI