Delving into Out-of-Distribution Detection with Medical Vision-Language Models
This work addresses the underexplored yet critical challenge of out-of-distribution (OOD) detection for medical vision-language models (VLMs) operating on highly variable and noisy clinical imaging data. We present the first systematic study of OOD detection in medical VLMs. Our method introduces a hierarchical prompting mechanism to enhance cross-modal semantic discrimination and establishes a comprehensive, multi-faceted OOD evaluation framework covering both semantic and covariate shifts. Integrating zero-shot inference, prompt engineering, and OOD confidence calibration, our approach enables distributional distance modeling and semantic alignment across diverse general-purpose and domain-specific medical VLMs. Extensive experiments on multiple medical imaging benchmarks demonstrate significant improvements over existing VLM-based OOD detection methods. The code is publicly released, establishing a new paradigm for trustworthy AI in healthcare.