Controllable Traffic Simulation through LLM-Guided Hierarchical Chain-of-Thought Reasoning

2024-09-23

arXiv.org

AI Summary

Existing traffic simulation methods exhibit weak responsiveness to complex semantic instructions and poor controllability, hindering safety validation of autonomous driving systems across diverse scenarios. To address this, we propose the first LLM-guided hierarchical chain-of-thought mechanism, integrated with a geometry-aware cost function formulated in the Frenet coordinate system, enabling joint controllable generation of semantic descriptions and motion geometry. Our method synergistically combines diffusion models, large language models, hierarchical reasoning, and Frenet-based motion modeling. Evaluated on the Waymo Open Motion Dataset, it achieves a 23% improvement in generation efficiency and a 31% increase in semantic–geometric alignment accuracy over state-of-the-art diffusion-based approaches. The framework establishes a novel simulation paradigm for autonomous driving validation—characterized by high fidelity, interpretability, and strong controllability.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Evaluating autonomous driving systems in complex and diverse traffic scenarios through controllable simulation is essential to ensure their safety and reliability. However, existing traffic simulation methods face challenges in their controllability. To address this, this paper proposes a novel diffusion-based and LLM-enhanced traffic simulation framework. Our approach incorporates a unique chain-of-thought (CoT) mechanism, which systematically examines the hierarchical structure of traffic elements and guides LLMs to thoroughly analyze traffic scenario descriptions step by step, enhancing their understanding of complex situations. Furthermore, we propose a Frenet-frame-based cost function framework that provides LLMs with geometrically meaningful quantities, improving their grasp of spatial relationships in a scenario and enabling more accurate cost function generation. Experiments on the Waymo Open Motion Dataset (WOMD) demonstrate that our method handles more intricate descriptions, generates a broader range of scenarios in a controllable manner, and outperforms existing diffusion-based methods in terms of efficiency.

Problem

• Enhancing controllability in traffic simulation for autonomous driving evaluation
• Improving LLM understanding of complex traffic scenarios through hierarchical reasoning
• Generating accurate cost functions for better spatial relationship modeling

Innovation

• Diffusion-based and LLM-enhanced traffic simulation framework
• Hierarchical high-level understanding and low-level refinement modules
• Frenet-frame-based cost function for spatial relationships