Search papers, labs, and topics across Lattice.
This paper introduces HEFT, a teleoperation framework designed for full-size humanoids that effectively manages the challenges of tracking and payload interaction. By leveraging Privileged Motion Guidance (PMG) and a Windowed Payload Curriculum (WPC), HEFT enables robust motion tracking even under the constraints of noisy VR references and significant payloads. The framework was successfully implemented on a 175cm, 65kg humanoid robot, demonstrating effective tracking of various motions while carrying loads up to 24kg.
HEFT enables full-size humanoids to perform complex movements with heavy payloads, overcoming significant challenges in motion tracking and stability.
General motion tracking and teleoperation offer a promising path to scalable humanoid skill acquisition, yet most existing frameworks are validated on compact platforms or without real payload interaction, leaving full-size humanoids with real payloads largely unexplored. Scaling to full-size humanoids introduces two compounding challenges: their larger inertia and tighter balance margins make tracking highly sensitive to noise, drift, and retargeting errors from commodity VR trackers, while their payload potential remains largely underutilized. We present HEFT, a heavy-payload full-size humanoid teleoperation framework that addresses both challenges. HEFT learns from deployable noisy VR references with physically plausible reconstructed references through Privileged Motion Guidance (PMG), and uses a Windowed Payload Curriculum (WPC) with expert-guided payload caps to acquire robust heavy-payload tracking. We deploy HEFT on L7, a 175cm, 65kg humanoid. The robot tracks motions including turns, forward/backward locomotion, and squats under payloads up to 24kg.
Tsinghua AI