Beyond Calibration: Confounding Pathology Limits Foundation Model Specificity in Abdominal Trauma CT

2026-02-10

AI Summary

This study addresses the significant drop in specificity of foundation models for detecting bowel injuries in abdominal trauma CT scans, primarily due to negative-class heterogeneity—such as co-occurring solid organ injuries—which hinders clinical deployment. The authors systematically evaluate MedCLIP (zero-shot), RadDINO (linear probe), and several task-specific models on a multicenter RSNA dataset for rare bowel injury detection. Foundation models achieve AUCs of 0.64–0.68 and sensitivities of 79–91%, but exhibit low specificities of only 33–50%, with a further decline exceeding 50 percentage points in patients with concurrent injuries, markedly underperforming task-specific models. The work identifies negative-class heterogeneity as the primary driver of poor specificity and demonstrates that supervised fine-tuning with labeled data effectively mitigates this limitation.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Other Foundations of Computer Vision
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Purpose: Translating foundation models into clinical practice requires evaluating their performance under compound distribution shift, where severe class imbalance coexists with heterogeneous imaging appearances. This challenge is relevant for traumatic bowel injury, a rare but high-mortality diagnosis. We investigated whether specificity deficits in foundation models are associated with heterogeneity in the negative class. Methods: This retrospective study used the multi-institutional, RSNA Abdominal Traumatic Injury CT dataset (2019-2023), comprising scans from 23 centres. Two foundation models (MedCLIP, zero-shot; RadDINO, linear probe) were compared against three task-specific approaches (CNN, Transformer, Ensemble). Models were trained on 3,147 patients (2.3% bowel injury prevalence) and evaluated on an enriched 100-patient test set. To isolate negative-class effects, specificity was assessed in patients without bowel injury who had concurrent solid organ injury (n=58) versus no abdominal pathology (n=50). Results: Foundation models achieved equivalent discrimination to task-specific models (AUC, 0.64-0.68 versus 0.58-0.64) with higher sensitivity (79-91% vs 41-74%) but lower specificity (33-50% vs 50-88%). All models demonstrated high specificity in patients without abdominal pathology (84-100%). When solid organ injuries were present, specificity declined substantially for foundation models (50-51 percentage points) compared with smaller reductions of 12-41 percentage points for task-specific models. Conclusion: Foundation models matched task-specific discrimination without task-specific training, but their specificity deficits were driven primarily by confounding negative-class heterogeneity rather than prevalence alone. Susceptibility to negative-class heterogeneity decreased progressively with labelled training, suggesting adaptation is required before clinical implementation.

Problem

• foundation model
• specificity
• negative-class heterogeneity
• abdominal trauma CT
• confounding pathology

Innovation

• foundation models
• negative-class heterogeneity
• confounding pathology
• specificity
• abdominal trauma CT