Problem
Research questions and friction points this paper is trying to address.
Automate test scenario generation for autonomous vehicles
Mitigate LLM hallucinations in traffic rule processing
Validate and synthesize executable simulation scripts
TARGET: Automated Scenario Generation from Traffic Rules for Testing Autonomous Vehicles via Validated LLM-Guided Knowledge Extraction
🤖 AI Summary
To address the high cost and low coverage of manually constructed scenarios in autonomous driving safety testing, this paper proposes a method for automatically generating executable test scenarios from natural-language traffic regulations. We introduce a domain-specific language (DSL)-constrained large language model (LLM) knowledge extraction paradigm, integrating syntactic and semantic validation mechanisms with simulation script synthesis to enable modular, verifiable rule parsing and scenario generation—effectively mitigating LLM hallucination. Evaluated on seven mainstream autonomous driving systems using 284 rule-derived test scenarios, our approach detected 610 violations and collisions; all test executions produced replayable logs, and two critical defects were confirmed by developers. The method significantly improves testing efficiency and ensures comprehensive regulatory coverage while maintaining traceability and reproducibility.
📝 Abstract
Recent incidents with autonomous vehicles highlight the need for rigorous testing to ensure safety and robustness. Constructing test scenarios for autonomous driving systems (ADSs), however, is labor-intensive. We propose TARGET, an end-to-end framework that automatically generates test scenarios from traffic rules. To address complexity, we leverage a Large Language Model (LLM) to extract knowledge from traffic rules. To mitigate hallucinations caused by large context during input processing, we introduce a domain-specific language (DSL) designed to be syntactically simple and compositional. This design allows the LLM to learn and generate test scenarios in a modular manner while enabling syntactic and semantic validation for each component. Based on these validated representations, TARGET synthesizes executable scripts to render scenarios in simulation. Evaluated seven ADSs with 284 scenarios derived from 54 traffic rules, TARGET uncovered 610 rule violations, collisions, and other issues. For each violation, TARGET generates scenario recordings and detailed logs, aiding root cause analysis. Two identified issues were confirmed by ADS developers: one linked to an existing bug report and the other to limited ADS functionality.
Innovation
Methods, ideas, or system contributions that make the work stand out.
LLM-guided knowledge extraction from traffic rules
Domain-specific language for modular scenario generation
Syntactic and semantic validation for scenario components
🔎 Similar Papers
No similar papers found.
