GCR: Geometry-Consistent Routing for Task-Agnostic Continual Anomaly Detection

2026-01-05

arXiv.org

AI Summary

This work addresses the challenge of task-agnostic continual anomaly detection, where category identities are unknown and dynamically expand over time. Conventional methods suffer from unstable routing and performance collapse due to incomparable anomaly scores across categories. To overcome this, we propose a lightweight multi-expert framework that decouples routing from anomaly scoring for the first time: routing is performed in a frozen pre-trained visual feature space based on distances to nearest prototypes, while anomaly scoring follows a standard prototype-based approach. By introducing a geometrically consistent prototype matching strategy, our method resolves cross-category score incomparability without requiring end-to-end retraining. Experiments on MVTec AD and VisA demonstrate near-zero forgetting, stable expert routing, and consistently strong anomaly detection and localization performance.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Feature-based anomaly detection is widely adopted in industrial inspection due to the strong representational power of large pre-trained vision encoders. While most existing methods focus on improving within-category anomaly scoring, practical deployments increasingly require task-agnostic operation under continual category expansion, where the category identity is unknown at test time. In this setting, overall performance is often dominated by expert selection, namely routing an input to an appropriate normality model before any head-specific scoring is applied. However, routing rules that compare head-specific anomaly scores across independently constructed heads are unreliable in practice, as score distributions can differ substantially across categories in scale and tail behavior. We propose GCR, a lightweight mixture-of-experts framework for stabilizing task-agnostic continual anomaly detection through geometry-consistent routing. GCR routes each test image directly in a shared frozen patch-embedding space by minimizing an accumulated nearest-prototype distance to category-specific prototype banks, and then computes anomaly maps only within the routed expert using a standard prototype-based scoring rule. By separating cross-head decision making from within-head anomaly scoring, GCR avoids cross-head score comparability issues without requiring end-to-end representation learning. Experiments on MVTec AD and VisA show that geometry-consistent routing substantially improves routing stability and mitigates continual performance collapse, achieving near-zero forgetting while maintaining competitive detection and localization performance. These results indicate that many failures previously attributed to representation forgetting can instead be explained by decision-rule instability in cross-head routing. Code is available at https://github.com/jw-chae/GCR

Problem

• task-agnostic
• continual anomaly detection
• expert selection
• routing
• cross-head score comparability

Innovation

• geometry-consistent routing
• task-agnostic continual anomaly detection
• mixture-of-experts
• prototype-based scoring
• routing stability