Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift

2026-07-11

AI Summary

The robustness of foundation models under out-of-distribution (OOD) domain shifts in mammography remains unclear. This study presents the first systematic benchmark evaluating 15 vision and vision-language foundation models—including Mammo-FM, MaMA, and DINOv3—across 12 OOD datasets using a unified protocol of frozen backbones with linear probing. Models are trained on three source datasets and assessed for generalization across tasks involving breast density estimation, BI-RADS categorization, and cancer detection. Results reveal that breast-specific vision-language models achieve the highest average performance but exhibit substantial dataset-level heterogeneity; the general-purpose model DINOv3 demonstrates notably robust performance; and pretraining on mammography data does not consistently enhance OOD generalization. These findings underscore the critical importance of dataset-level OOD evaluation in medical imaging.

Abstract

Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear. We benchmark 15 foundation-model backbones across breast density, BI-RADS severity, and cancer status using a unified frozen-backbone linear-probe protocol, training on 3 source datasets and evaluating on 12 task-compatible out-of-distribution (OOD) datasets after label harmonization. Mammography-specific vision-language models (Mammo-FM and MaMA) provide the strongest mean OOD performance, but robustness is not explained by mammography exposure alone. DINOv3 remains a competitive vision-only baseline, and mammography-adapted pretraining does not consistently improve generalization. Dataset-level analysis further shows that even leading models show heterogeneous performance across datasets. Feature-space inspection reveals that useful representations can preserve clinical signal while retaining dataset and acquisition structure. These findings highlight dataset-level OOD evaluation as a central criterion for assessing mammography representations. Our code is publicly available: https://github.com/biomedia-mira/mammo-ood.

Problem

• foundation models
• mammography
• domain shift
• out-of-distribution
• robustness

Innovation

• foundation models
• domain shift
• mammography
• out-of-distribution robustness
• linear probing