Beyond Metadata: CAPRA for Hidden Subgroup Analysis under Missing Metadata in Medical Imaging

2026-07-10

AI Summary

This work addresses the challenge that deployment of medical imaging models often masks performance disparities across critical subpopulations due to missing demographic, acquisition, and quality metadata, while existing robust learning methods struggle to uncover latent subgroup structures. To this end, the authors propose CAPRA, a framework that predicts semantic axes from images and performs patient-level cross-fitted calibration using only a small number of samples with metadata. This yields an interpretable and reusable subgroup interface that enables failure analysis and downstream robust learning even in the absence of metadata. Notably, CAPRA is the first method to calibrate and reveal real-world failure modes without requiring explicit subgroup labels or relying solely on raw image slices. Experiments on fundus, dermoscopy, and chest X-ray datasets demonstrate its ability to expose previously overlooked performance gaps and enhance robustness for specific subgroups.

Abstract

Medical imaging models are often deployed without the demographic, acquisition, and quality metadata needed for subgroup auditing. Once those metadata disappear, clinically critical failure modes can be masked by strong aggregate performance, and many robust-learning methods lose the group structure they rely on. We present CAPRA, a calibrated proxy-axis framework for hidden subgroup analysis under missing metadata. CAPRA predicts image-derived semantic axes, calibrates axis posteriors on a small metadata-labeled split via patient-level cross-fitting, and organizes those posteriors into a calibrated subgroup interface that supports both deployment-time failure analysis and downstream robust learning without requiring subgroup labels at deployment. Across fundus, dermoscopy, and chest radiography, CAPRA reveals disparity patterns missed by metadata-only slicing, remains informative under dataset shift, and produces subgroup partitions that align more closely with explicit failure axes than image-only or latent-slice baselines. The same interface can also be reused by downstream robust learners, although those gains are domain-dependent. Overall, CAPRA turns hidden subgroup analysis under missing metadata into a calibrated, interpretable, and reusable subgroup interface for deployment-time analysis and robust transfer.

Problem

• missing metadata
• hidden subgroup analysis
• medical imaging
• subgroup auditing
• failure modes

Innovation

• CAPRA
• hidden subgroup analysis
• missing metadata
• calibrated proxy-axis
• robust learning