Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving

2026-06-22

AI Summary

Existing end-to-end autonomous driving approaches lack explicit evaluation of the future consequences of planned trajectories, leading to insufficient reliability in complex dynamic scenarios. This work proposes IRR-Drive, a novel framework that deeply integrates an adaptive multimodal reflection mechanism into the planning pipeline for the first time. By generating an initial driving intent and predicting future semantic bird’s-eye-view (BEV) representations, IRR-Drive models scene evolution within a dual-modality reflective space combining textual and BEV modalities, enabling scene-complexity-aware self-correction and trajectory refinement. The method synergistically unifies vision–language–action modeling, future BEV prediction, and reflection-guided training, augmented by an adaptive reflection-based reward mechanism. Evaluated on the NAVSIM benchmark, IRR-Drive achieves state-of-the-art performance on both PDMS and EPDMS metrics, demonstrating its effectiveness.

Abstract

Recent Vision-Language-Action (VLA) models have advanced end-to-end autonomous driving by incorporating reasoning for better interpretability and planning quality. However, most existing approaches directly generate the final trajectory without explicitly examining its future consequences, which limits their reliability in complex and dynamic environments. To address this limitation, we propose IRR-Drive (Intend, Reflect, Refine), an adaptive multimodal reflection framework for autonomous driving. Specifically, to tightly couple high-level reasoning with physical constraints, IRR-Drive first generates a preliminary textual intention and anticipates potential interactions by predicting future semantic bird's-eye view (BEV) representations. This dual-modality (Text + BEV) reflection space explicitly models anticipated scene evolution, enabling the model to rigorously self-correct and refine its initial intent before generating the final trajectory. Furthermore, to balance planning performance and computational efficiency, we construct reflection-oriented training data and design an adaptive reflection reward, enabling the model to adaptively select its reasoning mode according to scene complexity. Instead of using reasoning primarily as an auxiliary interpretation, IRR-Drive directly integrates an adaptive reflection mechanism into the planning framework, enabling grounded, decision-aware trajectory correction that is driven by scene complexity. Our method achieves state-of-the-art performance on the NAVSIM benchmark in both PDMS and EPDMS. Extensive experiments demonstrate the effectiveness of our multimodal reflection framework and validate the efficacy of the proposed adaptive reflection strategy.

Problem

• autonomous driving
• trajectory planning
• reasoning
• multimodal reflection
• scene complexity

Innovation

• multimodal reflection
• adaptive reasoning
• vision-language-action (VLA)
• bird's-eye view (BEV) prediction
• autonomous driving planning