Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution

2025-10-13

AI Summary

Existing end-to-end autonomous driving approaches predominantly adopt single-frame or short-horizon temporal paradigms, neglecting long-term spatiotemporal evolution of driving scenes—leading to insufficient decision adaptability in complex scenarios. This paper introduces SeerDrive, the first end-to-end framework establishing a bidirectional closed-loop relationship between trajectory planning and scene evolution: it predicts future bird’s-eye-view (BEV) representations to model environmental dynamics and jointly optimizes them with future vehicle states in an iterative manner, enabling perception-aware closed-loop decision-making. Key innovations include a future-BEV-guided cooperative planning mechanism and an iterative scene–vehicle co-optimization architecture. SeerDrive achieves significant improvements over state-of-the-art methods on NAVSIM and nuScenes, demonstrating both the effectiveness and generalizability of explicitly modeling long-horizon scene dynamics.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

End-to-end autonomous driving methods aim to directly map raw sensor inputs to future driving actions such as planned trajectories, bypassing traditional modular pipelines. While these approaches have shown promise, they often operate under a one-shot paradigm that relies heavily on the current scene context, potentially underestimating the importance of scene dynamics and their temporal evolution. This limitation restricts the model's ability to make informed and adaptive decisions in complex driving scenarios. We propose a new perspective: the future trajectory of an autonomous vehicle is closely intertwined with the evolving dynamics of its environment, and conversely, the vehicle's own future states can influence how the surrounding scene unfolds. Motivated by this bidirectional relationship, we introduce SeerDrive, a novel end-to-end framework that jointly models future scene evolution and trajectory planning in a closed-loop manner. Our method first predicts future bird's-eye view (BEV) representations to anticipate the dynamics of the surrounding scene, then leverages this foresight to generate future-context-aware trajectories. Two key components enable this: (1) future-aware planning, which injects predicted BEV features into the trajectory planner, and (2) iterative scene modeling and vehicle planning, which refines both future scene prediction and trajectory generation through collaborative optimization. Extensive experiments on the NAVSIM and nuScenes benchmarks show that SeerDrive significantly outperforms existing state-of-the-art methods.

Problem

• Modeling bidirectional relationship between trajectory planning and scene evolution
• Overcoming one-shot paradigm limitations in autonomous driving systems
• Enhancing adaptive decision-making through future-aware closed-loop optimization

Innovation

• Bidirectional modeling of trajectory and scene evolution
• Future-aware planning using predicted BEV features
• Iterative scene modeling and vehicle planning optimization