Search papers, labs, and topics across Lattice.
This paper introduces Teleopit, a comprehensive humanoid teleoperation system that integrates full-body motion, dexterous hand control, and viewpoint manipulation without the need for specialized wearable sensors. By employing a history encoder and failure-aware rewind sampling, the system enhances motion tracking accuracy for both motion-capture and live VR inputs. The optimization-based hand retargeter effectively maps human hand movements to various dexterous hands, achieving impressive task success rates of 90.0% and 95.0% in real-robot applications using trained policies.
Teleopit achieves 95% task success in humanoid teleoperation by seamlessly mapping human body and hand signals to a robotic counterpart without specialized sensors.
Humanoid teleoperation for demonstration collection requires coordinated whole-body motion, continuous dexterous hand control, and viewpoint control. Existing systems either simplify hand commands or depend on dedicated wearable sensors for fine-grained hand motion. We introduce Teleopit, a full-embodiment teleoperation system that maps body, hand, and head signals from VR to a humanoid body, configurable dexterous hands, and a 2-DoF active vision module. A history encoder and failure-aware rewind sampling improve the motion tracker on both motion-capture and live VR references. An optimization-based hand retargeter combines normalized finger directions, fingertip closure, and thumb-frame alignment to map human hand motion to different dexterous hands without tuning hand-specific objective or solver hyperparameters. Component experiments evaluate tracking success rate and retargeting behavior, while real-robot teleoperation demonstrates coordinated locomotion, manipulation, and viewpoint control. ACT and GR00T N1.7 policies trained on 96 successful demonstrations collected with Teleopit achieve task success rates of 90.0% and 95.0%, respectively, when deployed on the humanoid. The project page is available at https://botrunner64.github.io/teleopit-page.
