What Matters for Scalable and Robust Learning in End-to-End Driving Planners?

2026-03-16

AI Summary

This work addresses the challenge that existing end-to-end autonomous driving methods, while performing well on open-loop datasets, struggle to balance scalability and robustness in closed-loop settings. The authors systematically evaluate the impact of key design choices—including high-resolution perceptual representations, decoupled trajectory modeling, and generative planning—on closed-loop performance. They propose BevAD, a lightweight end-to-end framework based purely on behavioral cloning, which integrates bird’s-eye-view feature grids, high-fidelity perception, and a decoupled trajectory generation mechanism. Evaluated on the Bench2Drive benchmark, BevAD achieves a 72.7% success rate and demonstrates strong data-scaling capabilities, thereby revealing both the limitations of prevailing architectures in closed-loop scenarios and the potential for synergistic optimization across perception and planning components.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

End-to-end autonomous driving has gained significant attention for its potential to learn robust behavior in interactive scenarios and scale with data. Popular architectures often build on separate modules for perception and planning connected through latent representations, such as bird's eye view feature grids, to maintain end-to-end differentiability. This paradigm emerged mostly on open-loop datasets, with evaluation focusing not only on driving performance, but also intermediate perception tasks. Unfortunately, architectural advances that excel in open-loop often fail to translate to scalable learning of robust closed-loop driving. In this paper, we systematically re-examine the impact of common architectural patterns on closed-loop performance: (1) high-resolution perceptual representations, (2) disentangled trajectory representations, and (3) generative planning. Crucially, our analysis evaluates the combined impact of these patterns, revealing both unexpected limitations as well as underexplored synergies. Building on these insights, we introduce BevAD, a novel lightweight and highly scalable end-to-end driving architecture. BevAD achieves 72.7% success rate on the Bench2Drive benchmark and demonstrates strong data-scaling behavior using pure imitation learning. Our code and models are publicly available here: https://dmholtz.github.io/bevad/

Problem

• end-to-end driving
• closed-loop performance
• scalable learning
• robustness
• autonomous driving

Innovation

• end-to-end driving
• closed-loop performance
• BEV representation
• scalable imitation learning
• generative planning