ForSim: Stepwise Forward Simulation for Traffic Policy Fine-Tuning

2026-02-02

AI Summary

This work addresses the covariate shift and limited multimodal behavioral realism inherent in open-loop imitation learning for traffic simulation by proposing a stepwise closed-loop forward simulation paradigm. At each virtual time step, the method integrates physics-driven motion dynamics to select the candidate trajectory best aligned with the reference trajectory in both space and time for propagation, while simultaneously updating predictions for other agents to achieve interaction-aware and temporally consistent multi-agent evolution. By introducing a stepwise closed-loop mechanism into forward simulation for the first time, the approach effectively balances multimodal diversity with intra-modal consistency. Integrated into the RIFT framework, it demonstrates significant improvements in safety, efficiency, realism, and comfort, thereby validating the critical role of closed-loop multimodal interaction modeling in enhancing the fidelity of autonomous driving simulation.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

As the foundation of closed-loop training and evaluation in autonomous driving, traffic simulation still faces two fundamental challenges: covariate shift introduced by open-loop imitation learning and limited capacity to reflect the multimodal behaviors observed in real-world traffic. Although recent frameworks such as RIFT have partially addressed these issues through group-relative optimization, their forward simulation procedures remain largely non-reactive, leading to unrealistic agent interactions within the virtual domain and ultimately limiting simulation fidelity. To address these issues, we propose ForSim, a stepwise closed-loop forward simulation paradigm. At each virtual timestep, the traffic agent propagates the virtual candidate trajectory that best spatiotemporally matches the reference trajectory through physically grounded motion dynamics, thereby preserving multimodal behavioral diversity while ensuring intra-modality consistency. Other agents are updated with stepwise predictions, yielding coherent and interaction-aware evolution. When incorporated into the RIFT traffic simulation framework, ForSim operates in conjunction with group-relative optimization to fine-tune traffic policy. Extensive experiments confirm that this integration consistently improves safety while maintaining efficiency, realism, and comfort. These results underscore the importance of modeling closed-loop multimodal interactions within forward simulation and enhance the fidelity and reliability of traffic simulation for autonomous driving. Project Page: https://currychen77.github.io/ForSim/

Problem

• traffic simulation
• covariate shift
• multimodal behaviors
• forward simulation
• autonomous driving

Innovation

• stepwise forward simulation
• closed-loop simulation
• multimodal behavior
• traffic policy fine-tuning
• interaction-aware agents