Dataset-Origin Signatures and Shortcut Learning in Screening Mammography AI: A Cross-Dataset Case Study

2026-07-16

AI Summary

This study demonstrates that incorporating externally sourced biopsy-confirmed positive cases into low-prevalence mammographic screening data induces significant performance degradation due to domain shift, with AUC-ROC dropping from 0.737 to 0.620–0.644. Despite uniform preprocessing, strong dataset-specific signatures persist across sources, causing models to learn spurious correlations with dataset origin rather than genuine pathological signals. To investigate this, the authors employ an EfficientNet-B5 backbone initialized with Mammo-CLIP weights as a frozen linear probe and evaluate it under patient-level cross-dataset splits. The severity of domain shift is further corroborated by near-perfect accuracy in predicting dataset provenance. These findings underscore the necessity of domain-aware strategies when integrating heterogeneous mammography datasets to mitigate shortcut learning and improve generalization.

Abstract

Reliable AI for screening mammography requires training data representative of the low cancer prevalence and subtle abnormalities found in screening populations. We examined whether supplementing such data with biopsy-confirmed cases from abnormal-enriched external datasets improves performance. Using the Newfoundland and Labrador Breast Screening Dataset (NLBSD) alongside CBIS-DDSM and CMMD, we evaluated an EfficientNet-B5 encoder initialized with Mammo-CLIP weights as a frozen linear probe under consistent preprocessing and patient-level splits. The NLBSD-only model achieved an AUC-ROC of 0.737 (95% CI [0.686, 0.785]). Adding external positive cases reduced performance in every configuration (AUC-ROC = 0.620--0.644; DeLong test, Holm-corrected $p < 0.05$), with degradation increasing as additional sources were introduced. Domain-matched evaluation produced modest gains only when the training and test domains coincided, and no configuration surpassed the NLBSD-only model. As a diagnostic, we reframed the task as predicting each examination's dataset of origin. The datasets were separated almost perfectly despite identical preprocessing, indicating that dataset-specific characteristics strongly influence the learned representation. These findings show that naïvely pooling abnormal-enriched mammography datasets can introduce domain shift that outweighs the benefit of additional positive cases. Differences in acquisition, intensity mapping, and dataset construction persist after normalization, motivating domain-aware strategies for combining heterogeneous mammography datasets.

Problem

• dataset-origin signatures
• shortcut learning
• domain shift
• mammography AI
• cross-dataset generalization

Innovation

• dataset-origin signatures
• shortcut learning
• domain shift
• mammography AI
• cross-dataset generalization