MaRS: Robust Out-of-Distribution Detection via Mahalanobis Residual Scoring

2026-06-21

AI Summary

This work addresses the vulnerability of foundation models in medical imaging to distribution shifts caused by variations in patients, acquisition devices, or scanning conditions, which can severely compromise their reliability. To mitigate this issue, we propose a lightweight, post-hoc out-of-distribution (OOD) detection method that leverages an autoencoder to learn the in-distribution data manifold and introduces Mahalanobis distance in the reconstruction residual space for variance-aware anomaly scoring. We demonstrate that the performance bottleneck of existing approaches stems not from reconstruction fidelity but from the use of simplistic reconstruction error metrics for scoring. Our key insight is the first application of Mahalanobis distance in residual space, which preserves anisotropic structural information critical for accurate OOD detection. Extensive experiments across three medical imaging modalities, diverse distribution shift scenarios, and models of varying scales consistently show substantial improvements over state-of-the-art baselines based on confidence, distance, and reconstruction metrics.

Abstract

Foundation models provide highly descriptive representations for medical images, yet their reliability degrades under distribution shifts arising from changes in patients, devices, or acquisition conditions. Reliable out-of-distribution (OOD) detection is therefore essential for safe deployment. Recent post-hoc detectors efficiently exploit frozen embeddings (\emph{e.g.}, kNN), whereas reconstruction-based OOD detection in latent feature space has seen limited adoption due to inconsistent performance. In this work, we show that the limitation of reconstruction-based methods in latent space does not stem from poor reconstruction quality, but from how reconstruction errors are scored. Standard $L_2$ residual norms collapse the anisotropic residual structure, thereby suppressing informative deviations. To address this limitation, we introduce \texttt{MaRS} (Mahalanobis Residual Scoring), a label-free OOD detector that learns an in-distribution manifold using a lightweight autoencoder and measures deviation via a Mahalanobis distance on reconstruction residuals, yielding variance-aware OOD scores. Across three imaging modalities, multiple types of distribution shift, and different model families and scales, \texttt{MaRS} outperforms established confidence-, distance-, and reconstruction-based baselines, while remaining fully post-hoc and lightweight. The code is available at https://github.com/francescodisalvo05/mars.

Problem

• out-of-distribution detection
• distribution shift
• medical imaging
• foundation models
• OOD detection

Innovation

• Mahalanobis distance
• out-of-distribution detection
• reconstruction residual
• foundation models
• medical image analysis