Layer Selection in VLMs for Zero-Shot OOD Detection via Multi-Resolution Entropy Estimation

2026-09-08

AI Summary

研究解决了医学AI系统中零样本OOD检测问题，通过多分辨率熵估计方法选择最佳中间层表示，提高检测稳定性与性能。

Abstract

Out-of-distribution (OOD) detection is crucial for safe deployment of medical AI systems, where domain shifts arise across institutions, acquisition protocols, and patient populations. VLMs enable zero-shot OOD detection by embedding images into a language-aligned latent space, where cross-modal similarity serves as a non-parametric confidence signal for identifying in-distribution samples. Yet existing methods rely almost exclusively on final-layer embeddings, implicitly assuming that the deepest representations are universally optimal. We first show that this assumption does not hold in medical imaging: intermediate layers provide complementary OOD signals, and the optimal representational depth depends on the respective image modality. While prior work selects layer combinations via entropy minimization of normalized histograms, we demonstrate that single-resolution entropy estimation is highly sensitive to binning choices, leading to performance variations of up to 19.3% AUROC. To address this instability, we propose a multi-resolution entropy estimation strategy that aggregates histogram statistics across multiple discretization scales, enabling robust and stable intermediate-layer selection. Across two medical OOD benchmarks, namely MIDOG and OASIS, covering distinct imaging modalities, diverse shift types, and different VLM backbones, our method consistently outperforms state-of-the-art approaches, offering a lightweight and stable solution for zero-shot OOD detection.

Problem

• Out-of-distribution Detection
• Visual-Language Models
• Medical Imaging
• Zero-shot Learning

Innovation

• multi-resolution entropy estimation
• intermediate-layer selection
• zero-shot OOD detection
• medical imaging