Search papers, labs, and topics across Lattice.
This paper introduces TAGA, a Terrain-aware Active Gaze learning framework that enhances humanoid locomotion by integrating vision, proprioception, and motion commands to selectively focus on relevant terrain features. The framework enables the model to learn anticipatory cues through reinforcement learning, significantly improving training efficiency without additional supervision. Key results include robust and generalizable locomotion capabilities in both simulation and hardware, achieving the largest reported real-world gap traversal distance of 1.2m while maintaining stability under challenging conditions.
Gaze behaviors learned through reinforcement alone can lead to unprecedented humanoid locomotion capabilities, including a record 1.2m gap traversal.
Agile humanoid locomotion across diverse challenging terrain demands both wide perceptual coverage and precise local geometry understanding. Motivated by the way humans selectively look at relevant terrain during locomotion, we introduce TAGA, a Terrain-aware Active Gaze learning framework for Attention-based humanoid control. By fusing vision, proprioception, and motion commands, our framework guides the model to learn anticipatory cues and actively attend to specific areas of the height scan, selectively using these informative regions for the downstream network. This adaptively increases the information density of observations under tight onboard computational constraints, thus enabling fine-grained perceptive locomotion over larger-scale terrains. We find that such gaze behaviors can naturally emerge through reinforcement learning alone, without requiring additional supervision or explicit guidance, significantly improve training efficiency. As a result, the trained policy demonstrates robust and generalizable locomotion in simulation and on hardware, including reliable terrain-aware foothold selection, elevated-platform traversal, competitive sparse-foothold traversal, and the largest reported real-world gap traversal distance of 1.2m among perceptive humanoid locomotion systems, while maintaining stability under severe perceptual disturbances and environmental interference.
NUS