From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning

2026-08-10

AI Summary

This work addresses the limited deep semantic understanding and multidimensional reasoning capabilities of existing traffic anomaly detection methods. To this end, we propose the first multitask video–language reasoning framework tailored for traffic anomalies, along with TAR—a large-scale dataset featuring chain-of-thought annotations—and the associated evaluation benchmark, TAR-Bench. Built upon the MAVEN architecture, our model integrates multiscale video evidence to jointly generate structured event descriptions, question–answer pairs, and reasoning trajectories. Multitask fine-tuning substantially enhances the model’s holistic reasoning capacity. Experimental results demonstrate that a model jointly fine-tuned on ten tasks achieves a 21.4-point improvement over zero-shot baselines on TAR-Bench, effectively bridging the gap between question-answering accuracy and spatiotemporal reasoning performance.

Abstract

We present TAR (Traffic Anomaly Reasoning) and TAR-Bench datasets, resources for training and evaluating video-language models beyond anomaly detection. TAR contains 44,040 chain-of-thought training annotations across 10 tasks for 3,670 CCTV videos ($\sim$26 hours) from eight public datasets. Its evaluation component, TAR-Bench, contains 960 human-curated test annotations for 80 held-out clips trimmed from 17 public YouTube videos. TAR's training annotations are produced with MAVEN, which consolidates multi-scale video evidence into structured event descriptions before generating question-answer pairs and reasoning traces. On TAR-Bench, eleven vision-language models reveal that strong question-answering accuracy does not reliably predict temporal or scene reasoning ability. Multi-task fine-tuning on TAR yields consistent gains, with the full 10-task model improving aggregate score by 21.4 points over its zero-shot baseline. TAR and TAR-Bench provide the official training and in-domain evaluation data for AI City Challenge 2026 Track 3. The dataset is available at https://huggingface.co/datasets/nvidia/PhysicalAI-Traffic-Anomaly-Reasoning

Problem

• Traffic Anomaly Reasoning
• Video-Language Models
• Multi-Task Reasoning
• Temporal Reasoning
• Scene Understanding

Innovation

• Traffic Anomaly Reasoning
• Chain-of-Thought Annotation
• Multi-Task Video-Language Learning
• TAR-Bench
• MAVEN