Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
Addressing the scarcity of coordinated body-hand robot data, WB-WAM unifies body, root, and dexterous hand trajectories into a shared physical action space for generative video pre-training across 1,880 hours of heterogeneous human and robot data. The resulting representations are refined via task-aligned PICO mid-training on retargeted egocentric demonstrations and adapted to physical platforms using auxiliary forward kinematics supervision. The method achieves an 81.9% success rate in the HumanoidArena simulation benchmark and an 84.0% mean success rate across five real-world loco-manipulation tasks while significantly reducing reliance on physical robot demonstrations.
Co-training generative world models on 1,880 hours of retargeted human motion and video unlocks an 84% real-world humanoid loco-manipulation success rate while drastically slashing the need for costly teleoperated robot data.
Humanoid loco-manipulation demands coordinated body and hand behavior, while conventional robot pre-training data provide limited coverage of such whole-body motion. We present WB-WAM, a World Action Model that incorporates explicit whole-body action supervision into generative video pre-training. A shared physical action space integrates body, root, and dexterous hand annotations from heterogeneous sources, enabling joint video and action learning from 1880.2 hours of partially annotated video and motion data. The resulting priors are refined through PICO mid-training and adapted to robot tasks with auxiliary forward kinematics supervision. We construct WB-Datasets to support these stages with retargeted egocentric human demonstrations and robot trajectories, allowing task-aligned human motion to supplement limited robot data. Evaluations in simulation demonstrate strong whole-body task performance with 81.9% in HumanoidArena, while real-world experiments further validate WB-WAM with 84.0% mean success across five tasks. Moreover, task-aligned PICO mid-training improves downstream task performance while reducing the need for real-robot demonstrations. These results support heterogeneous whole-body pre-training and human motion transfer as a practical route to data-efficient humanoid loco-manipulation.