OmniSCS: Omni Safety-Critical Scenario Synthesis for Autonomous Driving via a Fully Editable Driving World

2026-07-07

AI Summary

Existing methods struggle to balance data fidelity and the efficiency of generating safety-critical scenarios (SCS) when editing agent states in autonomous driving scenes. This work proposes a fully editable driving world framework that enables flexible manipulation at both appearance and trajectory levels through dual-policy agent reconstruction, depth-optimized background reconstruction, object insertion, and trajectory editing, while preserving high physical and visual fidelity. Evaluated on the nuScenes, Waymo, and KITTI datasets, the proposed approach significantly outperforms current state-of-the-art methods and supports real-time closed-loop simulation at 13 Hz, thereby enhancing the safety and robustness of autonomous driving systems.

Abstract

The synthesis of safety-critical scenarios (SCS) and their evaluation through closed-loop simulations are crucial for developing robust autonomous driving systems. A key aspect of this process involves editing agent states in both appearance and trajectory levels within existing scenes. However, current methods struggle to preserve data fidelity after scene editing and fail to efficiently generate high-quality SCS through such modifications. To overcome these limitations, we propose OmniSCS, an innovative system that generates photorealistic SCS with high physical fidelity while enabling closed-loop testing in synthetic environments. OmniSCS comprises two key modules: 1) A Fully Editable Driving World Construction module that maintains high-fidelity agent appearance and background during scene editing via dual-strategy agent reconstruction and depth-refinement background reconstruction methods. 2) A SCS Synthesis module that facilitates object insertion and agent trajectory editing to synthesize diverse SCS while preserving data fidelity. Experiments on nuScenes, Waymo, and KITTI datasets show that OmniSCS outperforms state-of-the-art methods in edited scene fidelity. We further validate its ability to enhance autonomous driving algorithms and support real-time (13Hz) closed-loop testing. Overall, OmniSCS provides a safer, more effective, and cost-efficient solution for SCS optimization and testing in autonomous driving.

Problem

• safety-critical scenarios
• scene editing
• data fidelity
• autonomous driving
• scenario synthesis

Innovation

• Safety-Critical Scenario Synthesis
• Fully Editable Driving World
• Photorealistic Simulation
• Closed-Loop Testing
• High-Fidelity Scene Editing