Long-term Traffic Simulation via Structured Autoregressive Modeling

2026-06-30

AI Summary

This work addresses the challenges of modeling persistent multi-agent interactions and handling dynamic agent counts in long-horizon traffic simulation by proposing RosettaSim, a unified framework that leverages structured autoregressive modeling to encode scene topology, agent states, and intentions into variable-length sequences. By integrating the attention mechanism of a frozen small-scale large language model with motion distribution consistency, RosettaSim achieves high-accuracy short-term prediction and stable long-term generation. Additionally, the authors introduce a retrieval-augmented traffic evaluation metric, RTE, which substantially enhances semantic alignment between simulated and real-world traffic behaviors. On the WOSAC benchmark, RosettaSim attains state-of-the-art performance in both short- and long-horizon simulation, while RTE demonstrates a correlation of 0.83 with standard metrics—surpassing existing methods that achieve only 0.74.

Abstract

Interactive traffic simulation is a vital world model for autonomous driving. A central challenge in long-horizon simulation is modeling sustained multi-agent interactions, which is further exacerbated by dynamic token cardinality as agents continuously enter and exit the scene. In this work, we propose that the solution lies in the synergy between the architectural inductive biases and statistical priors of large-scale sequence models, e.g., Large Language Models (LLMs). Our probing experiments reveal that the transferability of attention mechanisms and the distributional consistency between motion tokens and natural language enable small-scale, heavily frozen LLMs to rapidly adapt to traffic modeling. Building on this insight, we introduce RosettaSim, a unified framework that projects scene topology, agent states, and spawning intents into a structured autoregressive stream with variable length, achieving both strong short-term accuracy and stable long-horizon simulation fidelity. Furthermore, evaluating extended rollouts presents yet another hurdle, as one-to-one agent correspondence inevitably fades over time. To address this, we introduce Retrieval-based Traffic Evaluation (RTE), which retrieves semantically similar real-world scenarios as context-aware reference anchors. Experiments on the Waymo Open Sim Agent Challenge (WOSAC) demonstrate that RosettaSim achieves state-of-the-art performance in both short- and long-term simulation. Furthermore, RTE exhibits a stronger correlation with standard metrics ($r=0.83$) than existing approaches ($r=0.74$), indicating improved alignment with long-horizon simulation fidelity.

Problem

• long-horizon simulation
• multi-agent interactions
• dynamic token cardinality
• traffic simulation evaluation

Innovation

• Structured Autoregressive Modeling
• Large Language Models (LLMs)
• Long-horizon Traffic Simulation
• Retrieval-based Traffic Evaluation (RTE)
• Multi-agent Interaction