Using Diffusion Ensembles to Estimate Uncertainty for End-to-End Autonomous Driving

2025-05-31

AI Summary

End-to-end autonomous driving trajectory planning often neglects uncertainty modeling or relies on hand-crafted uncertainty representations, limiting safety and generalization. To address this, we propose EnDfuser—the first framework to integrate ensemble diffusion models into end-to-end driving planning. Given a single-frame multimodal input (camera + LiDAR), EnDfuser directly generates 128 diverse candidate trajectories, explicitly modeling the posterior trajectory distribution without predefined uncertainty parameterization. Our method unifies diffusion Transformers, cross-modal attention pooling, and trajectory-level denoising sampling to enable plug-and-play uncertainty estimation. Evaluated on the CARLA Longest6 benchmark, EnDfuser achieves a driving score of 70.1, operates near real-time, and significantly enhances planning robustness and safety margins—particularly in complex, dynamic scenarios.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

End-to-end planning systems for autonomous driving are improving rapidly, especially in closed-loop simulation environments like CARLA. Many such driving systems either do not consider uncertainty as part of the plan itself, or obtain it by using specialized representations that do not generalize. In this paper, we propose EnDfuser, an end-to-end driving system that uses a diffusion model as the trajectory planner. EnDfuser effectively leverages complex perception information like fused camera and LiDAR features, through combining attention pooling and trajectory planning into a single diffusion transformer module. Instead of committing to a single plan, EnDfuser produces a distribution of candidate trajectories (128 for our case) from a single perception frame through ensemble diffusion. By observing the full set of candidate trajectories, EnDfuser provides interpretability for uncertain, multi-modal future trajectory spaces, where there are multiple plausible options. EnDfuser achieves a competitive driving score of 70.1 on the Longest6 benchmark in CARLA with minimal concessions on inference speed. Our findings suggest that ensemble diffusion, used as a drop-in replacement for traditional point-estimate trajectory planning modules, can help improve the safety of driving decisions by modeling the uncertainty of the posterior trajectory distribution.

Problem

• Estimating uncertainty in autonomous driving trajectory planning
• Leveraging fused perception data for end-to-end driving systems
• Improving safety by modeling multi-modal trajectory distributions

Innovation

• Uses diffusion model for trajectory planning
• Combines attention pooling and trajectory planning
• Generates multiple candidate trajectories via ensemble