Event Stream based Multi-Modal Video Anomaly Detection: A Benchmark Dataset and Algorithms

2026-07-10

AI Summary

This work addresses the limitations of visible-light video-based anomaly detection under challenging conditions such as illumination variations, rapid motion, and complex backgrounds. To overcome these issues, the authors propose a multimodal anomaly detection framework that integrates event camera data with conventional visible-light video, leveraging the high temporal resolution and motion sensitivity of event streams to enhance robustness. The key contributions include the introduction of TJUTCM Pha, the first large-scale visible-light–event anomaly detection benchmark dataset; a contrastive multimodal pretraining strategy that aligns textual, visual, and event semantics; and an adaptive spatiotemporal feature fusion mechanism. Extensive experiments demonstrate that the proposed method significantly outperforms existing approaches across multiple established benchmarks and the newly curated dataset, validating the efficacy of event-aware sensing for real-world anomaly detection.

Abstract

Video anomaly detection (VAD) is critical for automated surveillance but remains fragile under challenging conditions such as illumination variations, fast motion, and complex backgrounds when relying solely on visible light videos. To address these limitations, we propose EVAD, an event enhanced VAD framework that jointly exploits conventional video and event streams captured by bio inspired event cameras. Event sensors asynchronously capture brightness changes with high temporal resolution, offering robustness to motion blur and extreme lighting, and providing motion salient cues complementary to video based visual information. To support multi modal VAD research, we construct a large scale visible event benchmark comprising 6.3 billion events and 376,368 video frames collected under diverse illumination levels, motion patterns, and background complexities, filling the gap of realistic and scalable datasets for event based anomaly detection. Building upon this dataset, we design a contrastive multi modal pretraining framework to learn discriminative event representations by aligning semantic embeddings across event streams, visible videos, and textual descriptions. An adaptive fusion module then dynamically integrates event based temporal cues with video based spatial semantics, improving robustness to environmental disturbances. Experiments on benchmarks and the proposed TJUTCM Pha dataset demonstrate that E VAD consistently outperforms methods, validating the effectiveness of event-based sensing for VAD in real world scenarios.

Problem

• Video Anomaly Detection
• Event Stream
• Multi-Modal
• Illumination Variations
• Motion Blur

Innovation

• event-based vision
• video anomaly detection
• multi-modal fusion
• contrastive pretraining
• benchmark dataset