SynAD: Enhancing Real-World End-to-End Autonomous Driving Models through Synthetic Data Integration

2025-10-28

AI Summary

Limited coverage of real-world driving scenarios constrains the generalization and safety of end-to-end autonomous driving models. To address this, we propose a novel paradigm integrating synthetic data: (1) for the first time, leveraging the optimal-information agent from multi-agent simulation as the ego vehicle; (2) designing a Map-to-BEV network that directly generates bird’s-eye-view (BEV) features from high-definition maps—bypassing reliance on synthetic sensor inputs; and (3) introducing path-level scenario projection and joint synthetic-real training. Experiments demonstrate substantial improvements in model robustness and safety under unseen conditions, achieving state-of-the-art performance across multiple benchmarks. Our approach establishes a scalable, high-fidelity data augmentation pathway for end-to-end autonomous driving, effectively bridging the reality gap while preserving semantic and geometric fidelity.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Recent advancements in deep learning and the availability of high-quality real-world driving datasets have propelled end-to-end autonomous driving. Despite this progress, relying solely on real-world data limits the variety of driving scenarios for training. Synthetic scenario generation has emerged as a promising solution to enrich the diversity of training data; however, its application within E2E AD models remains largely unexplored. This is primarily due to the absence of a designated ego vehicle and the associated sensor inputs, such as camera or LiDAR, typically provided in real-world scenarios. To address this gap, we introduce SynAD, the first framework designed to enhance real-world E2E AD models using synthetic data. Our method designates the agent with the most comprehensive driving information as the ego vehicle in a multi-agent synthetic scenario. We further project path-level scenarios onto maps and employ a newly developed Map-to-BEV Network to derive bird's-eye-view features without relying on sensor inputs. Finally, we devise a training strategy that effectively integrates these map-based synthetic data with real driving data. Experimental results demonstrate that SynAD effectively integrates all components and notably enhances safety performance. By bridging synthetic scenario generation and E2E AD, SynAD paves the way for more comprehensive and robust autonomous driving models.

Problem

• Enhancing autonomous driving with synthetic data integration
• Addressing limited scenario diversity in real-world training data
• Bridging synthetic scenarios and end-to-end autonomous driving models

Innovation

• Uses synthetic multi-agent scenarios for ego vehicle
• Projects path scenarios onto maps for BEV features
• Integrates map-based synthetic data with real data