When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations

2026-06-15

AI Summary

This work addresses the challenge that deep neural networks tend to overgeneralize under distribution shifts, while existing out-of-distribution (OOD) detection methods lack semantic interpretability—limiting their reliability in high-stakes medical applications. The authors propose a novel approach based on class-conditional semantic perturbations: leveraging sparse autoencoders trained on in-distribution data to learn class-specific concept vectors, which are then used to induce semantic perturbations in deep representations. By analyzing the resulting impact on class logits, the method evaluates prediction stability to discriminate OOD samples. This is the first framework to integrate concept vectors with representation perturbation, offering not only effective OOD detection on medical imaging tasks but also human-interpretable semantic rationales, thereby significantly enhancing model trustworthiness for clinical deployment.

Abstract

Deep neural networks have achieved remarkable performance across medical imaging tasks, yet their tendency to overgeneralize under distributional shifts poses a major obstacle to safe clinical deployment. Out-of-Distribution (OOD) detection methods aim to mitigate this risk, but most existing approaches rely on opaque internal signals with poorly understood semantic meaning, limiting trust in safety-critical settings. In this work, we propose an interpretable OOD detection framework that probes the stability of model predictions under class-conditioned semantic perturbations. Leveraging sparse autoencoders (SAEs), we learn class-specific concept vectors from in-distribution data that disentangle dense intermediate representations into sparse, semantically meaningful components. At inference, we perturb deeper-layer representations using the concept vectors associated with the model's predicted class and measure the class logits stability. We hypothesize that in-distribution samples exhibit low sensitivity to such perturbations, as their representations align with class-specific semantic directions, whereas OOD samples show amplified deviations due to representational misalignment. By framing OOD detection as a concept conditioned stability analysis, our approach provides both a discriminative OOD signal and an interpretable lens into the internal mechanisms driving model uncertainty, making it particularly suitable for high stakes medical applications.

Problem

• Out-of-Distribution Detection
• Interpretability
• Distributional Shift
• Medical Imaging
• Model Uncertainty

Innovation

• interpretable OOD detection
• representation perturbation
• sparse autoencoders
• concept vectors
• semantic stability