Continual Unsupervised Out-of-Distribution Detection

2024-06-04

arXiv.org

AI Summary

Existing unsupervised out-of-distribution (U-OOD) detection methods assume OOD samples arise from static, globally complementary distributions—overlooking the critical real-world scenario where deployed models continuously encounter task-relevant, temporally evolving unknown samples. To bridge this gap, we propose **Iterative Deployment Exposure (IDE)**, a novel continual U-OOD detection paradigm aligned with clinical deployment requirements. Methodologically, we design a **Mahalanobis–nearest-neighbor hybrid scoring function**, and introduce a **confidence-weighted few-shot OOD detector** enabling online calibration and lightweight model updates. Evaluated across multiple medical and general-purpose vision benchmarks, IDE consistently outperforms state-of-the-art U-OOD and continual learning approaches, demonstrating superior robustness to distribution shifts, adaptability to emerging OOD classes, and practical deployability in resource-constrained, dynamic environments.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Diffusion Models for Vision

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Deep learning models excel when the data distribution during training aligns with testing data. Yet, their performance diminishes when faced with out-of-distribution (OOD) samples, leading to great interest in the field of OOD detection. Current approaches typically assume that OOD samples originate from an unconcentrated distribution complementary to the training distribution. While this assumption is appropriate in the traditional unsupervised OOD (U-OOD) setting, it proves inadequate when considering the place of deployment of the underlying deep learning model. To better reflect this real-world scenario, we introduce the novel setting of continual U-OOD detection. To tackle this new setting, we propose a method that starts from a U-OOD detector, which is agnostic to the OOD distribution, and slowly updates during deployment to account for the actual OOD distribution. Our method uses a new U-OOD scoring function that combines the Mahalanobis distance with a nearest-neighbor approach. Furthermore, we design a confidence-scaled few-shot OOD detector that outperforms previous methods. We show our method greatly improves upon strong baselines from related fields.

Problem

• Detects out-of-distribution images in real-world deployment scenarios
• Improves unsupervised OOD detection with iterative refinement
• Addresses performance degradation in medical imaging diagnostics

Innovation

• Iterative Deployment Exposure for realistic U-OOD detection
• Combines Mahalanobis distance with nearest-neighbor scoring
• Confidence-scaled few-shot OOD detector for limited examples