TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding

2026-08-26

AI Summary

为解决交通异常理解问题，提出TAU-Agent框架，通过检索增强和视觉感知工具协作，结合视频帧和查询生成最终解释。

Abstract

Traffic Anomaly Understanding (TAU) requires models and systems to detect, reason about, and explain anomalous events in transportation videos. To address this challenge, we propose TAU-Agent, an agentic retrieval-augmented framework for traffic anomaly understanding. Given a task query, a central retrieval agent orchestrates two visual perception tools, namely a Video Captioning Tool and an Open-Vocabulary Tracking Tool, to retrieve and select query-relevant evidence, including captions, temporal intervals, and object trajectories. The selected evidence, together with sampled video frames and the input query, is provided to a supervised fine-tuned vision-language model for final reasoning and answer generation. We evaluate TAU-Agent on both the in-domain and the out-of-domain benchmarks from the AI City Challenge 2026. TAU-Agent achieves scores of 0.6779 on Track 3, 0.3998 on Track 7, and 67.9275 on Track 8, ranking second, twelfth, and fifth, respectively. Code is available at: https://github.com/siri-rouser/TAU-Agent.

Problem

• Traffic Anomaly Understanding
• Anomalous Events
• Transportation Videos

Innovation

• Agentic Retrieval-Augmented Framework
• Traffic Anomaly Understanding
• Visual Perception Tools
• Supervised Fine-Tuned Vision-Language Model