Search papers, labs, and topics across Lattice.
The paper introduces RealDexUMI, a novel wearable manipulation interface designed to enhance dexterous robot learning by providing a shared dexterous end-effector module that integrates a lightweight hand, in-hand vision, and fingertip tactile sensing. This system allows for real-time, intuitive teleoperation without the loss of dexterity, enabling precise control and seamless data collection that closely mirrors the actual deployment conditions. In experiments across eight complex manipulation tasks, policies trained on data from RealDexUMI achieved an impressive average success rate of 88.75%, demonstrating strong generalization to new poses and effective transfer across different robot embodiments.
Achieving an 88.75% success rate in dexterous manipulation tasks, RealDexUMI bridges the gap between human demonstrations and robot execution without losing critical dexterity.
Learning dexterous manipulation requires demonstrations that preserve fine hand-object interactions while remaining executable at deployment. Existing pipelines either lose deployable dexterity through retargeting or embodiment conversion, or rely on robot-specific teleoperation that is costly to scale and often lacks intuitive, contact-aware control for dexterous data collection. We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module that integrates a lightweight dexterous hand, in-hand vision, and fingertip tactile sensing. A palm-side isomorphic teleoperation glove maps human finger inputs to robot-hand joint commands, enabling real-time, retargeting-free, intuitive, and precise hand control. The shared hand and sensing modules yield zero-gap end-effector data, with matched in-hand observations, tactile signals, contacts, and hand actions between collection and deployment. Across eight real-robot tasks spanning fine-grained, contact-rich, long-horizon, and bimanual manipulation, policies trained on RealDexUMI data achieve an average success rate of 88.75%, generalize to unseen initial poses, and transfer across three embodiments. Website: https://research.beingbeyond.com/realdexumi
Tsinghua AI