Search papers, labs, and topics across Lattice.
This paper introduces Vision-to-Traversability Adaptation (ViTA), a framework that adapts Vision Foundation Models (VFMs) like SAM2 for reliable traversability estimation in unstructured outdoor environments. ViTA addresses key challenges by injecting task-specific knowledge via learnable traversability prompts, handling annotation ambiguity through Perspective-Diversified Training, and bridging the semantic-traversability discrepancy by distilling geometric knowledge. Experiments across diverse datasets demonstrate that ViTA achieves state-of-the-art IoU and Precision while reducing false positives and improving cross-domain generalization.
Adapting vision foundation models with task-specific prompts and geometric knowledge distillation dramatically improves traversability estimation in challenging outdoor environments, outperforming existing methods in both accuracy and reliability.
Vision-based approaches have become the dominant paradigm for traversability estimation in unstructured outdoor environments, typically adapting vision foundation models (VFMs) via semantic segmentation supervision. However, this paradigm faces three fundamental challenges that undermine its reliability: the task-agnostic design of VFMs, the ambiguity of traversability annotations, and the discrepancy between semantic labels and physical safety. We propose Vision-to-Traversability Adaptation (ViTA), a framework that adapts VFMs for reliable traversability estimation, instantiated on SAM2. ViTA injects task-specific knowledge through learnable traversability prompts while preserving the VFM's cross-domain generalization. To handle annotation ambiguity, we introduce Perspective-Diversified Training, which estimates semantic uncertainty to suppress confident predictions at ambiguous boundaries. To bridge the semantic-traversability discrepancy, we distill geometric knowledge during training, enabling slope and elevation reasoning from RGB images alone at inference. The semantic and geometric outputs are fused into a continuous traversability score that reflects both semantic uncertainty and geometric risk. Evaluations across diverse domains, including challenging real-world off-road datasets, demonstrate that ViTA achieves state-of-the-art IoU and Precision with substantial false-positive reduction and strong cross-domain generalization.
