Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MAMMOTH, a novel end-to-end navigation policy designed for robust autonomous mobility in challenging off-road environments, addressing the limitations of existing methods that rely heavily on RGB inputs. By integrating multiple modalities—RGB, Thermal, 3D Pointcloud, and Ego Velocity—MAMMOTH employs a modality dropout training scheme to enhance generalization under conditions of sensor degradation and missing inputs. The results from extensive real-world experiments show that MAMMOTH significantly outperforms existing approaches in collision avoidance and terrain-aware planning, particularly in low-visibility scenarios.
MAMMOTH achieves robust off-road navigation by effectively handling missing sensor modalities, outperforming traditional RGB-reliant methods in challenging environments.
Reliable autonomous navigation in unstructured off-road environments remains a critical unsolved challenge due to extreme terrain diversity, drastic illumination variations and acute sensor degradation. Recent developments have approached the problem as a traversability costmap estimation or visual navigation task. However, many exhibit heavy reliance on RGB modality, leading to poor performance in varied illumination such as glares, shadows or low ambient light. Achieving robust generalization in such conditions requires integrating modalities that provide supplementary scene information. Such multi-modal methods suffer from a rigid dependency on the presence of near-perfect sensor inputs, leaving them unable to robustly handle sensor degradation or individual modality failure. To address these limitations, we introduce MAMMOTH (MAsking Multi-Modal inputs for Off-road Traversability Heuristic-informed navigation), a unified end-to-end navigation policy for robust off-road visual-goal-conditioned navigation and undirected exploration. Specifically, MAMMOTH efficiently fuses multi-modal observations (RGB, Thermal, 3D Pointcloud and Ego Velocity) and is trained with a modality dropout scheme, enabling it to generalize to missing modalities at inference time. Furthermore, we employ a diffusion policy to learn the joint conditional probability distribution of physically-grounded trajectories and a intrinsic traversability heuristic. MAMMOTH utilizes this heuristic to prefer safer, smoother trajectories. We validate MAMMOTH through extensive real-world robot experiments in distinct off-road environments, including night-time operation. Our results demonstrate superior performance, with significant improvements in collision avoidance, terrain-aware planning and generalization to missing modalities. The code and dataset used for this work will be made publicly available.