TrafficAlign: Aligning Large Language Models for Traffic Scenario Generation

2026-06-27

AI Summary

This work addresses the challenge that current large language models (LLMs) struggle to generate autonomous driving traffic scenarios aligned with real-world traffic distributions. To bridge this gap, the authors propose TrafficAlign, a novel framework that, for the first time, aligns LLMs using real-world driving videos from multiple geographic regions. The approach automatically parses real traffic videos, synthesizes realistic traffic scenarios, and validates the generated data to enhance distributional fidelity and realism. Experimental results demonstrate that scenarios produced by TrafficAlign uncover 10.8% more collision risks on average compared to baseline methods. Furthermore, fine-tuning autonomous driving models with these synthesized scenarios reduces collision rates by 36.1% and achieves strong distribution alignment across six diverse regional datasets.

Abstract

Recent research has investigated the use of large language models (LLMs) to generate traffic scenarios for autonomous driving. However, pretrained LLMs often fail to align with real-world traffic distributions. In this work, we present TrafficAlign, an automated framework that synthesizes traffic scenarios based on real-world driving videos, performs data validation, and aligns LLMs with the synthesized scenarios. The evaluation shows that traffic scenarios generated by TrafficAlign are highly effective, revealing up to 10.8% more collisions on average across three autonomous driving models than state-of-the-art methods. Furthermore, fine-tuning these driving models with TrafficAlign-generated scenarios significantly reduced collision rates by 36.1% compared with the original models. A qualitative study using traffic datasets from six geographically diverse regions shows that TrafficAlign-generated scenarios exhibit strong alignment with corresponding traffic distributions in these regions.

Problem

• large language models
• traffic scenario generation
• distribution alignment
• autonomous driving
• real-world traffic

Innovation

• Traffic scenario generation
• Large language models
• Distribution alignment
• Autonomous driving
• Data synthesis