Wasserstein-Aligned Localisation for VLM-Based Distributional OOD Detection in Medical Imaging

2026-05-06

AI Summary

This study addresses the challenge of zero-shot anomaly localization in medical imaging, where the scarcity of healthy anatomical context hinders the detection of rare pathologies. The authors reformulate the task as a contrastive inference problem grounded in the reference distribution of normal anatomy and propose WALDO, a training-free framework that leverages entropy-weighted slice Wasserstein distance, Goldilocks-region sampling, and self-consistency aggregation. For the first time, they reveal a non-monotonic relationship between reference similarity and localization accuracy from the perspective of distributional shift. Integrating DINOv2 features, weighted non-maximum suppression, and multimodal large language models (e.g., Qwen2.5-VL-72B), WALDO achieves an mAP@30 of 43.5% ± 1.6% on the NOVA brain MRI benchmark, outperforming existing zero-shot methods by 19% (p < 0.01).

Abstract

Zero-shot anomaly localisation via vision-language models (VLMs) offers a compelling approach for rare pathology detection, yet its performance is fundamentally limited by the absence of healthy anatomical context. We reformulate zero-shot localisation as a comparative inference problem in which anomalies are identified through structured comparison against reference distributions of normal anatomy. We introduce WALDO, a training-free framework grounded in optimal transport theory that enables comparative reasoning through: (i) entropy-weighted Sliced Wasserstein distances for anatomically-aware reference selection from DINOv2 patch distributions, (ii) Goldilocks zone sampling exploiting the non-monotonic relationship between reference similarity and localisation accuracy, and (iii) self-consistency aggregation via weighted non-maximum suppression. We theoretically analyse the Goldilocks effect through distributional divergence, and show that references with moderate similarity minimize a bias-variance trade-off in comparative visual reasoning. On the NOVA brain MRI benchmark, WALDO with Qwen2.5-VL-72B achieves $43.5_{\pm1.6}\%$ mAP@30 (95\% CI: [40.4, 46.7]), representing a 19\% relative improvement over zero-shot baselines. Cross-model evaluation shows consistent gains: GPT-4o achieves $32.0_{\pm6.5}\%$ and Qwen3-VL-32B achieves $32.0_{\pm6.6}\%$ mAP@30. Paired McNemar tests confirm statistical significance ($p<0.01$). Source code is available at https://github.com/bkainz/WALDO_MICCAI26_demo .

Problem

• zero-shot anomaly localisation
• medical imaging
• out-of-distribution detection
• vision-language models
• anatomical context

Innovation

• Wasserstein distance
• zero-shot anomaly localisation
• vision-language models
• optimal transport
• distributional OOD detection