Towards Accurate and Robust Surveillance Roadside IVD via Trackletized Audio-Visual Reasoning

2026-06-20

AI Summary

This work addresses the challenges of inaccurate and non-robust vehicle idling detection in roadside monitoring under domain shift and low data efficiency. To this end, the authors propose a multi-object tracking–guided audiovisual fusion method that associates detection outputs into tracklets and performs classification at the tracklet level to enhance signal-to-noise ratio and temporal stability. Furthermore, a tracklet-based audiovisual inference mechanism is introduced, which explicitly models the spatial alignment prior between vehicles and microphones, enabling detector-free cross-domain adaptation. Evaluated on the newly curated AVIVD-LT and AVIVD-M benchmarks, the proposed approach demonstrates superior deployment robustness and data efficiency across day-to-day and site-to-site transfer scenarios.

Abstract

Idling Vehicle Detection (IVD) seeks to determine, at the final frame of a video clip, whether any vehicle is idling, meaning the vehicle is stationary with its engine running, using synchronized video from a remote surveillance camera and multichannel audio captured by spatially distributed wireless microphones along the roadside. Prior full-image, clip-level fusion approaches tend to overfit scene background and full-frame context, produce unstable temporal decisions, and lack an explicit spatial prior to align vehicles with microphones, which makes them brittle under domain shift and data inefficient. Instead, we introduce TAVR-IVD, an audio-visual framework guided by multi-object tracking. Our method detects vehicles, links detections into tracklets, and classifies each vehicle by operating on its tracklet. This design raises the effective signal-to-noise ratio, stabilizes temporal decisions through tracklets, enforces an explicit spatial prior to align vehicles with microphones, and adapts across domains with limited calibration annotations while remaining detector agnostic and efficient. To evaluate deployment robustness, we further curate two evaluation extensions, AVIVD-LT and AVIVD-M, covering inter-day and cross-site shifts.

Problem

• Idling Vehicle Detection
• Audio-Visual Surveillance
• Domain Shift
• Spatial Alignment
• Temporal Stability

Innovation

• Tracklet-based reasoning
• Audio-visual fusion
• Idling Vehicle Detection
• Multi-object tracking
• Domain adaptation