PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving

2026-06-30

AI Summary

This work addresses the limitation of existing end-to-end autonomous driving approaches, which rely solely on instantaneous sensor inputs and lack the human driver’s experience-based anticipatory capability. To overcome this, the authors propose a memory-augmented architecture that integrates geospatial visual priors through dual memory modules and an adaptive memory gating mechanism. Guided by planned trajectories, the system enables spatial anticipation without dependence on real-time perception while retaining a safety fallback capability. Evaluated on the NAVSIM-v2 benchmark, the method significantly enhances the performance of multiple end-to-end baselines and demonstrates robust, reliable driving behavior even under sensor failure conditions, thereby improving overall system robustness.

Abstract

Most end-to-end autonomous driving methods rely solely on instantaneous sensor observations, limiting them to reactive behavior without the anticipatory foresight human drivers employ through prior experience. We introduce geospatial visual priors, street-level visual context anchored to the intended driving route, providing visual-spatial foresight independent of real-time sensors. We propose a memory augmentation module featuring a dual-memory architecture and an adaptive memory gate, which can be easily integrated into existing end-to-end approaches. This design pairs a contextual memory for retrieved priors with a persistent fallback memory, and dynamically regulates the influence of memories based on current state compatibility. Evaluated on the NAVSIM-v2 benchmark, our approach consistently improves performance across diverse end-to-end baselines. Furthermore, because these priors are independent of onboard sensors, our method inherently improves robustness against sensor corruption, while the dual-memory design ensures safe fallback when the retrieved priors themselves become unreliable. Our project page is available at https://ori-mrg.github.io/PriorEye.

Problem

• autonomous driving
• geospatial priors
• visual foresight
• sensor dependency
• anticipatory behavior

Innovation

• geospatial visual priors
• dual-memory architecture
• adaptive memory gate
• end-to-end autonomous driving
• sensor-agnostic robustness