Search papers, labs, and topics across Lattice.
This paper introduces World-Model-Augmented Visual Locomotion (WM-LOCO), a method that combines a recurrent world model with a PPO policy to enhance locomotion on foothold-constrained terrain. By leveraging a predictive summary of near-future observations and rewards based on proprioception and a single depth image, WM-LOCO significantly improves foot-placement decisions in challenging environments. In simulations, WM-LOCO outperforms a baseline on gaps and stepping stones and matches its success on stairs while achieving greater stride efficiency and reduced pelvis acceleration, demonstrating a 93.3% success rate in physical deployment on a humanoid robot.
A predictive world model enables humanoid robots to navigate challenging terrains with a staggering 93.3% success rate, outperforming traditional methods.
Foothold-constrained terrain is characterized by sparse, discontinuous, or geometrically restricted feasible foot contacts, as encountered on stepping stones, across gaps, and on narrow stair treads. On such terrain, a single misstep often leaves little room to recover, so policies that base foot-placement decisions primarily on the immediately visible terrain are prone to failure. We ask whether a learned predictive summary of near-future observations and rewards can provide the anticipatory information required in such settings. We present World-Model-Augmented Visual Locomotion (WM-LOCO), which jointly trains a recurrent world model and a PPO policy. Conditioned on proprioception and a single onboard depth image, the world model produces a predictive recurrent feature that guides the policy, without explicit foothold labels. In simulation, WM-LOCO succeeds on gaps and stepping stones where a matched baseline fails completely, and matches the baseline's success rate on stairs while improving stride efficiency and reducing pelvis acceleration. We deploy the same policy onboard a physical Unitree G1 humanoid using onboard proprioception and a single depth stream; it traverses all three terrain classes with an average success rate of 93.3%.
