Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces CART, a context-aware high-level controller for legged robots that integrates proprioception and exteroception to improve terrain understanding and adaptation. CART addresses the "Visual-Texture Paradox" by using a temporal sequence selection approach to fuse visual and proprioceptive data. Experiments on ANYmal-C (simulation) and SPOT (real-world) show CART improves stability by up to 45% in real-world scenarios and achieves a 5% higher success rate than baselines in simulation.
Legged robots can now walk more stably on complex terrain by learning to see and feel, overcoming the "Visual-Texture Paradox" that plagues vision-reliant methods.
Animals in nature combine multiple modalities, such as sight and feel, to perceive terrain and develop an understanding of how to walk on uneven terrain in a stable manner. Similarly, legged robots need to develop their ability to stably walk on complex terrains by developing an understanding of the relationship between vision and proprioception. Most current terrain adaptation methods are susceptible to failure on complex, off-road terrain as they rely on prior experience, particularly observations from a vision sensor. This experience-based learning often creates a Visual-Texture Paradox between what has been seen and how it actually feels. In this work, we introduce CART, a high-level controller built on a context-aware terrain adaptation approach that integrates proprioception and exteroception from onboard sensing to achieve a robust understanding of terrain. We evaluate our method on multiple terrains using an ANYmal-C robot on the IsaacSim simulator and a Boston Dynamics SPOT robot for our real-world experiments. To evaluate the learned contextual terrain properties, we adapt vibrational stability on the base of the robot as a metric. We compare CART with various state-of-the-art baselines equipped with multimodal sensing in both simulation and the real world. CART achieves an average success rate improvement of 5% over all baselines in simulation and improves the overall stability up to 45% and 24% in the real world without increasing the time taken by the robot to accomplish locomotion tasks.