Search papers, labs, and topics across Lattice.
This paper introduces Geo-Anchored Fine-Tuning (GAFT), a novel method designed to enhance hazard identification in off-road navigation by leveraging geometry-derived priors to fine-tune vision foundation models. The approach addresses the challenge of limited training data from rare failure events by aligning spatial attention maps with geometric cues, thus improving model generalization. Experimental results on a forest hazard benchmark show that GAFT significantly outperforms existing methods, achieving a mean $F_2$ score of 0.3757 and a peak score of 0.570, indicating a substantial improvement in hazard identification capabilities.
GAFT transforms rare off-road failure data into robust hazard identification, boosting model performance by over 500% in critical scenarios.
Off-road navigation can fail when physical structures induce irrecoverable states such as high-centering or entrapment, requiring human interventions. Identifying these structures is crucial, yet challenging. Such failure events are rare and costly to collect, resulting in limited training data. Moreover, the collected data associate frames with outcomes, but do not indicate the visual cues responsible for the failure. Learning directly from these data can therefore exploit scenario-specific visual cues, leading to poor generalization. We propose \textbf{Geo-Anchored Fine-Tuning (GAFT)}, a parameter-efficient method that adapts a vision foundation model with a geometry-derived prior. It guides LoRA adaptation by aligning a spatial attention-rollout map with the geometry prior, while preserving pretrained representations. On an intervention-verified forest hazard benchmark, across ten independently trained adaptations, GAFT consistently outperforms frozen DINOv2 and supervised PEFT baselines, improving the repeated leave-one-scenario-out mean $F_2$ from 0.0607 to 0.3757 with statistical significance under paired analysis. Within these independently trained models, the best-performing GAFT model achieves a repeated-LOSO $F_2$ of 0.570. Code and benchmark: https://github.com/Xu-Yanran/geo_anchored_fine_tuning
