MLE-UVAD: Minimal Latent Entropy Autoencoder for Fully Unsupervised Video Anomaly Detection

2026-03-24

AI Summary

This work addresses the challenging setting of single-scene, fully unsupervised video anomaly detection—where training videos contain a mixture of normal and anomalous events without any labels—by proposing an entropy-guided autoencoder. The method jointly optimizes reconstruction loss and a novel minimum latent entropy (MLE) loss, which minimizes the entropy of latent representations to encourage anomalous samples to cluster in high-density regions dominated by normal data. This mechanism amplifies the reconstruction discrepancy between normal and anomalous frames, enabling robust anomaly discrimination without requiring clean normal-video priors or annotations. Extensive experiments demonstrate that the proposed approach significantly outperforms existing unsupervised methods on two established benchmarks as well as a newly introduced driving dataset.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: Video Understanding & Activity Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

In this paper, we address the challenging problem of single-scene, fully unsupervised video anomaly detection (VAD), where raw videos containing both normal and abnormal events are used directly for training and testing without any labels. This differs sharply from prior work that either requires extensive labeling (fully or weakly supervised) or depends on normal-only videos (one-class classification), which are vulnerable to distribution shifts and contamination. We propose an entropy-guided autoencoder that detects anomalies through reconstruction error by reconstructing normal frames well while making anomalies reconstruct poorly. The key idea is to combine the standard reconstruction loss with a novel Minimal Latent Entropy (MLE) loss in the autoencoder. Reconstruction loss alone maps normal and abnormal inputs to distinct latent clusters due to their inherent differences, but also risks reconstructing anomalies too well to detect. Therefore, MLE loss addresses this by minimizing the entropy of latent embeddings, encouraging them to concentrate around high-density regions. Since normal frames dominate the raw video, sparse anomalous embeddings are pulled into the normal cluster, so the decoder emphasizes normal patterns and produces poor reconstructions for anomalies. This dual-loss design produces a clear reconstruction gap that enables effective anomaly detection. Extensive experiments on two widely used benchmarks and a challenging self-collected driving dataset demonstrate that our method achieves robust and superior performance over baselines.

Problem

• video anomaly detection
• fully unsupervised
• single-scene
• anomaly detection
• unlabeled video

Innovation

• Minimal Latent Entropy
• Unsupervised Video Anomaly Detection
• Autoencoder
• Reconstruction Gap
• Entropy-guided Learning