MultiDrive: A Co-Simulation Framework Bridging 2D and 3D Driving Simulation for AV Software Validation

2025-05-20

AI Summary

In autonomous driving motion planning verification, low-fidelity 2D simulation offers efficiency but suffers from geometric and dynamic distortion, whereas high-fidelity 3D simulation ensures realism at prohibitive computational cost—hindering their synergistic use. Method: This paper introduces the first multi-agent, cross-fidelity simulation framework enabling seamless 2D/3D co-simulation. It integrates procedural scenario generation, cross-platform experiment orchestration, and automated trajectory comparison to establish reversible, semantics-preserving mappings between fidelity levels and ensure behavioral consistency. Results: Experiments demonstrate that the framework significantly reduces simulation migration overhead, mitigates the sim-to-real gap, and effectively uncovers latent assumption violations and behavior-intent mismatches of benchmark planners under photorealistic conditions. By enabling rigorous, scenario-based cross-fidelity validation, it enhances both the effectiveness and credibility of motion planning verification.

Technology Category

• Multiagent Systems: Multiagent Planning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Scenario-based testing using simulations is a cornerstone of Autonomous Vehicles (AVs) software validation. So far, developers needed to choose between low-fidelity 2D simulators to explore the scenario space efficiently, and high-fidelity 3D simulators to study relevant scenarios in more detail, thus reducing testing costs while mitigating the sim-to-real gap. This paper presents a novel framework that leverages multi-agent co-simulation and procedural scenario generation to support scenario-based testing across low- and high-fidelity simulators for the development of motion planning algorithms. Our framework limits the effort required to transition scenarios between simulators and automates experiment execution, trajectory analysis, and visualization. Experiments with a reference motion planner show that our framework uncovers discrepancies between the planner's intended and actual behavior, thus exposing weaknesses in planning assumptions under more realistic conditions. Our framework is available at: https://github.com/TUM-AVS/MultiDrive

Problem

• Bridges 2D and 3D simulation for AV validation
• Reduces effort in transitioning scenarios between simulators
• Identifies discrepancies in motion planner behavior

Innovation

• Bridges 2D and 3D simulators for AV testing
• Uses co-simulation and procedural scenario generation
• Automates scenario transition and experiment execution