Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamics

2025-06-10

AI Summary

Existing generative vehicle trajectory models are predominantly evaluated using reconstruction error, lacking online validation of traffic compliance and safety within real-time microscopic simulation—and overlooking critical engineering issues such as red-light running and illegal parking. This paper proposes the first online evaluation framework for generated trajectories at signalized intersections: it integrates SUMO as a closed-loop microscopic simulator to enable online assessment under unseen traffic conditions; introduces traffic-rule-aware metrics—including red-light violation rate and illegal parking frequency; and establishes a quantitative analysis toolchain. Experiments reveal that state-of-the-art models, despite low reconstruction error, frequently generate unsafe and noncompliant trajectories. The proposed metrics significantly enhance detection of safety-critical failures, delivering a reproducible, interpretable, and engineering-deployable evaluation paradigm for trustworthy traffic AI. (149 words)

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements

Abstract

Traffic Intersections are vital to urban road networks as they regulate the movement of people and goods. However, they are regions of conflicting trajectories and are prone to accidents. Deep Generative models of traffic dynamics at signalized intersections can greatly help traffic authorities better understand the efficiency and safety aspects. At present, models are evaluated on computational metrics that primarily look at trajectory reconstruction errors. They are not evaluated online in a `live' microsimulation scenario. Further, these metrics do not adequately consider traffic engineering-specific concerns such as red-light violations, unallowed stoppage, etc. In this work, we provide a comprehensive analytics tool to train, run, and evaluate models with metrics that give better insights into model performance from a traffic engineering point of view. We train a state-of-the-art multi-vehicle trajectory forecasting model on a large dataset collected by running a calibrated scenario of a real-world urban intersection. We then evaluate the performance of the prediction models, online in a microsimulator, under unseen traffic conditions. We show that despite using ideally-behaved trajectories as input, and achieving low trajectory reconstruction errors, the generated trajectories show behaviors that break traffic rules. We introduce new metrics to evaluate such undesired behaviors and present our results.

Problem

• Evaluating generative models for traffic intersection dynamics
• Assessing traffic rule violations in trajectory predictions
• Developing metrics for traffic engineering-specific model performance

Innovation

• Multi-vehicle trajectory forecasting model
• Online microsimulator evaluation
• Traffic rule violation metrics