CERES: Critical-Event Reconstruction via Temporal Scene Graph Completion

2024-10-17

arXiv.org

AI Summary

Safety-critical scenario generation for autonomous vehicle (AV) testing remains time-consuming and lacks diversity. Method: This paper proposes a controllable simulation-scenario generation method based on Temporal Scene Graphs (TSGs). It introduces TSG completion for safety-critical event reconstruction—novelly incorporating a user-controllable dual-condition mechanism (action + criticality) and integrating graph neural networks with conditional link prediction, all grounded in real-world driving data to ensure semantic controllability and physical plausibility. Contribution/Results: The proposed model significantly outperforms baselines on TSG link prediction. Generated scenarios demonstrate high physical consistency and strong test compatibility when validated in commercial simulators, enabling on-demand synthesis of high-fidelity, diverse, and safety-critical AV test scenarios.

Technology Category

• Planning, Routing, and Scheduling: Temporal Planning
• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

This paper proposes a method for on-demand scenario generation in simulation, grounded on real-world data. Evaluating the behaviour of Autonomous Vehicles (AVs) in both safety-critical and regular scenarios is essential for assessing their robustness before real-world deployment. By integrating scenarios derived from real-world datasets into the simulation, we enhance the plausibility and validity of testing sets. This work introduces a novel approach that employs temporal scene graphs to capture evolving spatiotemporal relationships among scene entities from a real-world dataset, enabling the generation of dynamic scenarios in simulation through Graph Neural Networks (GNNs). User-defined action and criticality conditioning are used to ensure flexible, tailored scenario creation. Our model significantly outperforms the benchmarks in accurately predicting links corresponding to the requested scenarios. We further evaluate the validity and compatibility of our generated scenarios in an off-the-shelf simulator.

Problem

• Generates dynamic temporal scene graphs for AV testing.
• Addresses time-consuming manual scenario creation in simulations.
• Predicts relationships using a temporal GNN for diverse traffic scenarios.

Innovation

• Dynamic temporal scene graphs generation
• Temporal Graph Neural Network model
• On-demand scenario tailored to preferences