Search papers, labs, and topics across Lattice.
This paper introduces geospatial visual priors that enhance end-to-end autonomous driving systems by providing anticipatory foresight based on street-level visual context rather than solely relying on real-time sensor data. The proposed memory augmentation module features a dual-memory architecture that integrates contextual and persistent fallback memories, dynamically adjusting their influence based on the current driving state. Evaluated on the NAVSIM-v2 benchmark, the approach significantly improves performance across various baseline models while enhancing robustness against sensor failures.
Geospatial visual priors enable autonomous vehicles to anticipate driving scenarios, improving performance and safety even when sensor data is compromised.
Most end-to-end autonomous driving methods rely solely on instantaneous sensor observations, limiting them to reactive behavior without the anticipatory foresight human drivers employ through prior experience. We introduce geospatial visual priors, street-level visual context anchored to the intended driving route, providing visual-spatial foresight independent of real-time sensors. We propose a memory augmentation module featuring a dual-memory architecture and an adaptive memory gate, which can be easily integrated into existing end-to-end approaches. This design pairs a contextual memory for retrieved priors with a persistent fallback memory, and dynamically regulates the influence of memories based on current state compatibility. Evaluated on the NAVSIM-v2 benchmark, our approach consistently improves performance across diverse end-to-end baselines. Furthermore, because these priors are independent of onboard sensors, our method inherently improves robustness against sensor corruption, while the dual-memory design ensures safe fallback when the retrieved priors themselves become unreliable. Our project page is available at https://ori-mrg.github.io/PriorEye.
