Search papers, labs, and topics across Lattice.
This paper introduces a hierarchical framework for whole-body humanoid locomotion that combines a diffusion model for terrain-aware motion generation with a reinforcement learning-based motion tracker. The diffusion model predicts reference motions based on retargeted human data and terrain perception, while the RL tracker executes these motions on the humanoid. Closed-loop fine-tuning of the tracker with the frozen motion generator improves robustness, enabling successful deployment on a Unitree G1 robot across diverse terrains.
Humanoid robots can now traverse complex terrains with whole-body coordination thanks to a novel framework that blends diffusion-based motion generation with robust RL-based tracking.
Whole-body humanoid locomotion is challenging due to high-dimensional control, morphological instability, and the need for real-time adaptation to various terrains using onboard perception. Directly applying reinforcement learning (RL) with reward shaping to humanoid locomotion often leads to lower-body-dominated behaviors, whereas imitation-based RL can learn more coordinated whole-body skills but is typically limited to replaying reference motions without a mechanism to adapt them online from perception for terrain-aware locomotion. To address this gap, we propose a whole-body humanoid locomotion framework that combines skills learned from reference motions with terrain-aware adaptation. We first train a diffusion model on retargeted human motions for real-time prediction of terrain-aware reference motions. Concurrently, we train a whole-body reference tracker with RL using this motion data. To improve robustness under imperfectly generated references, we further fine-tune the tracker with a frozen motion generator in a closed-loop setting. The resulting system supports directional goal-reaching control with terrain-aware whole-body adaptation, and can be deployed on a Unitree G1 humanoid robot with onboard perception and computation. The hardware experiments demonstrate successful traversal over boxes, hurdles, stairs, and mixed terrain combinations. Quantitative results further show the benefits of incorporating online motion generation and fine-tuning the motion tracker for improved generalization and robustness.
NVIDIA