Is Training Necessary for Anomaly Detection?

2026-01-30

AI Summary

This work addresses the limitations of existing reconstruction-based methods in multi-class unsupervised anomaly detection, which suffer from a trade-off between fidelity and stability and rely on task-specific training. The authors propose Retrieval-based Anomaly Detection (RAD), a training-free approach that stores multi-level features of normal samples in memory and detects anomalies by performing cross-level retrieval matching on test image patches. RAD achieves state-of-the-art performance without any training, challenging the prevailing assumption that training is indispensable in this domain. Theoretically, the method establishes that the retrieval score upper-bounds the reconstruction residual. Extensive experiments demonstrate RAD’s superiority across four benchmarks—MVTec-AD, VisA, Real-IAD, and 3D-ADAM—with a single normal image yielding a pixel-level AUROC of 96.7% on MVTec-AD.

Technology Category

• Computer Vision: Image and Video Retrieval
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Current state-of-the-art multi-class unsupervised anomaly detection (MUAD) methods rely on training encoder-decoder models to reconstruct anomaly-free features. We first show these approaches have an inherent fidelity-stability dilemma in how they detect anomalies via reconstruction residuals. We then abandon the reconstruction paradigm entirely and propose Retrieval-based Anomaly Detection (RAD). RAD is a training-free approach that stores anomaly-free features in a memory and detects anomalies through multi-level retrieval, matching test patches against the memory. Experiments demonstrate that RAD achieves state-of-the-art performance across four established benchmarks (MVTec-AD, VisA, Real-IAD, 3D-ADAM) under both standard and few-shot settings. On MVTec-AD, RAD reaches 96.7\% Pixel AUROC with just a single anomaly-free image compared to 98.5\% of RAD's full-data performance. We further prove that retrieval-based scores theoretically upper-bound reconstruction-residual scores. Collectively, these findings overturn the assumption that MUAD requires task-specific training, showing that state-of-the-art anomaly detection is feasible with memory-based retrieval. Our code is available at https://github.com/longkukuhi/RAD.

Problem

• anomaly detection
• unsupervised learning
• training-free
• reconstruction
• multi-class

Innovation

• training-free
• retrieval-based anomaly detection
• memory-based retrieval
• unsupervised anomaly detection
• multi-class anomaly detection