Search papers, labs, and topics across Lattice.
This study explores the transfer of manipulation skills from humans to robots using a novel bridging action representation that focuses on relative wrist translations rather than traditional 6DoF hand poses. By leveraging abundant human action data, the researchers developed a 蟺_0-like vision-language-action model that incorporates interleaved action tokens and attention masking to improve skill transfer. The key finding demonstrates that this approach significantly enhances the effectiveness of transferring human manipulation knowledge to robots across various bi-manual tasks, outperforming previous methods that relied on noisy human action estimates.
Bridging human and robot manipulation through wrist translation leads to a dramatic improvement in skill transfer efficiency, outpacing traditional methods reliant on noisy hand poses.
We study whether we can learn novel manipulation skills from human actions to a bi-manual robot with parallel grippers. Human action data is cheap, abundant, and diverse, making it one of the most promising resources for scaling up robot learning. Yet transferring skills from humans to robots remains hard: most prior work treats humans as just another bi-manual 6DoF embodiment, where hand-pose estimates are noisy and the contact patterns of human fingers differ fundamentally from those of a parallel gripper. We argue that learning rotation-inclusive action signals from human data is therefore sub-optimal, and instead propose a bridging action representation: the relative wrist translation within the initial head-camera frame, an action space shared by humans and robots. To handle the potential absence of certain action components in different embodiments, we build a 蟺_0-like vision-language-action model with interleaved action tokens and attention masking. On a suite of novel bi-manual manipulation tasks, our bridging action transfers human manipulation knowledge to robots far more effectively than noisy 6DoF human actions and scales with the amount of human data.
