Search papers, labs, and topics across Lattice.
HumanoidMimicGen is introduced to address the challenge of data scarcity in humanoid loco-manipulation by automatically generating demonstrations. It adapts contact-rich whole-body skills from a few demonstrations to new states using locomotion and manipulation planning to ensure stability and collision avoidance. Experiments on a new benchmark show that policies co-trained with HumanoidMimicGen data outperform those trained only on real-world data by 20%.
Generating synthetic data for humanoid robots can boost loco-manipulation performance by 20% compared to relying solely on real-world data.
Imitation learning is a promising approach for training humanoid robots to both walk and manipulate, but it requires a large number of demonstrations, which are time-intensive and difficult to collect via teleoperation. Existing data-generation algorithms can automatically synthesize demonstrations for manipulators, but they are ineffective on humanoids because their high-dimensional composite action spaces involve arms, legs, and torsos. We present HumanoidMimicGen, a method for generating humanoid legged loco-manipulation data. Our method adapts contact-rich whole-body skills from a handful of source demonstrations to new states, generalizing across changes in object pose. By interleaving these single- and dual-arm skills with whole-body locomotion and manipulation planning, the method generates stable, collision-free data across diverse scenes and layouts. To evaluate our approach, we introduce a new simulated loco-manipulation benchmark containing nine diverse tasks that test humanoid loco-manipulation capabilities. There, we demonstrate that HumanoidMimicGen automatically generates large datasets for imitation learning and enables a systematic study of how data generation and policy learning decisions impact model performance. We show that whole-body visuomotor policies co-trained with data generated by HumanoidMimicGen outperform those trained only on real-world data by 20%.