TeraSim: Uncovering Unknown Unsafe Events for Autonomous Vehicles through Generative Simulation

2025-03-05

AI Summary

Traditional traffic simulators struggle to model complex human-vehicle interactions, while data-driven approaches suffer from limited long-horizon behavioral fidelity and insufficient diversity of safety-critical scenarios. This paper introduces the first high-fidelity traffic simulation framework that synergistically integrates generative modeling with interpretability-guided control, built upon diffusion models and multi-agent reinforcement learning, and designed for seamless integration with physics engines and autonomous vehicle (AV) software stacks. The method autonomously discovers unknown hazardous scenarios and synthesizes diverse, realistic safety-critical events involving both dynamic and static agents. It successfully uncovers latent vulnerabilities in multiple commercial AV systems and enables repeatable, statistically reliable collision-rate evaluation. Our core contribution lies in transcending the dual limitations of rule-based and purely data-driven paradigms—achieving, for the first time in generative simulation, simultaneous long-horizon behavioral realism and controllable, diverse generation of safety-critical events.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Web data generation and simulation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Traffic simulation is essential for autonomous vehicle (AV) development, enabling comprehensive safety evaluation across diverse driving conditions. However, traditional rule-based simulators struggle to capture complex human interactions, while data-driven approaches often fail to maintain long-term behavioral realism or generate diverse safety-critical events. To address these challenges, we propose TeraSim, an open-source, high-fidelity traffic simulation platform designed to uncover unknown unsafe events and efficiently estimate AV statistical performance metrics, such as crash rates. TeraSim is designed for seamless integration with third-party physics simulators and standalone AV stacks, to construct a complete AV simulation system. Experimental results demonstrate its effectiveness in generating diverse safety-critical events involving both static and dynamic agents, identifying hidden deficiencies in AV systems, and enabling statistical performance evaluation. These findings highlight TeraSim's potential as a practical tool for AV safety assessment, benefiting researchers, developers, and policymakers. The code is available at https://github.com/mcity/TeraSim.

Problem

• Uncover unknown unsafe events for autonomous vehicles
• Address limitations of traditional and data-driven simulators
• Enable comprehensive AV safety evaluation and performance metrics

Innovation

• Generative simulation for diverse safety-critical events
• Integration with third-party physics simulators
• High-fidelity traffic simulation for AV safety assessment