Learning Where and When: Patch-Based Spatiotemporal Localization in Weakly Supervised Video Anomaly Detection

2026-06-28

AI Summary

This work addresses the limitation of existing weakly supervised video anomaly detection methods, which primarily focus on temporal localization while lacking precise spatial awareness, thereby hindering interpretability in real-world applications. We propose a patch-based spatiotemporal anomaly localization framework that jointly models the temporal and spatial locations of anomalies using only video-level labels. Leveraging multiple instance learning, our approach infers region-level anomaly scores from grid-level patch features and introduces a novel neighborhood-aware Top-k spatiotemporal selection strategy to generate fine-grained spatial anomaly maps without requiring bounding box supervision. Additionally, we provide frame-level bounding box annotations for two widely used datasets. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches across multiple benchmarks, achieving substantial improvements in spatiotemporal localization accuracy. The code, models, and new annotations are publicly released.

Abstract

Weakly supervised video anomaly detection (WSVAD) has predominantly focused on temporal localization, identifying when anomalies occur while largely neglecting their spatial extent within frames. Yet, spatial localization is essential for interpretability and practical deployment in real-world settings. We introduce a patch-based spatiotemporal framework for weakly supervised anomaly localization that jointly models where and when anomalies occur. Our approach operates on grid-level patch features and learns region-level anomaly scores under a multiple instance learning paradigm. We further propose a Proximity-Aware Top-k spatiotemporal selection strategy that enables the model to generate fine-grained spatial anomaly maps without requiring bounding-box supervision during training. Our method surpasses existing state-of-the-art approaches across multiple benchmarks, yielding substantial gains in spatiotemporal localization accuracy. In addition, we release frame-level bounding-box annotations for the test sets of two widely used datasets, along with our code and pretrained models, providing new resources to facilitate future research in spatially grounded WSVAD.

Problem

• Weakly Supervised Video Anomaly Detection
• Spatiotemporal Localization
• Spatial Localization
• Temporal Localization
• Anomaly Detection

Innovation

• patch-based
• spatiotemporal localization
• weakly supervised video anomaly detection
• multiple instance learning
• Proximity-Aware Top-k