🤖 AI Summary
This study addresses the challenges of unreliable probability calibration, inadequate uncertainty estimation, and the absence of a clinically actionable triage mechanism for thyroid nodule ultrasound images under distribution shift. To this end, the authors propose a deterministic classification framework based on a five-member deep ensemble, integrating a ConvNeXt-Tiny backbone, Squeeze-and-Excitation attention, member-wise vector scaling calibration, and mutual information–driven selective prediction to enable region-of-interest–level triage into three pathways: biopsy-free, biopsy-recommended, and radiologist review. On internal testing, the model achieves an AUC of 0.9395 (ECE = 0.0088), with a negative predictive value of 98.3% and a malignancy capture rate of 99.83% at a 50% retention rate in the biopsy-free pathway. External validation reveals a performance drop to an AUC of 0.7870, highlighting limitations in calibration threshold transferability while underscoring the method’s potential and challenges in high-stakes clinical settings.
📝 Abstract
Background: Deep learning models can classify thyroid nodules on ultrasound, but reliable clinical decision support also requires calibrated probabilities, uncertainty estimation, and selective referral, particularly under dataset shift.
Methods: We developed a calibrated deterministic five-member deep ensemble for ROI-based thyroid nodule classification and selective image-based triage. TN5000 was used for model development, five-fold cross-validation, member-wise vector-scaling calibration, and fold-specific threshold selection. TN3K served as an independent external dataset-shift evaluation. The framework used ConvNeXt-Tiny with squeeze-and-excitation attention, ensemble-mean malignancy probability, and mutual information (MI) as an ensemble-disagreement score. A three-tier policy assigned images to No-FNA suggestion, FNA recommendation, or radiologist review.
Results: On pooled out-of-fold TN5000 predictions, the ensemble achieved AUC-ROC 0.9395, AP 0.9715, ECE 0.0088, and Brier score 0.0813. At 50% nominal MI retention, 7.2% of cases received a No-FNA suggestion, 39.9% an FNA recommendation, and 52.9% radiologist review, with 98.3% No-FNA NPV and 99.83% malignancy capture. On TN3K, AUC-ROC decreased to 0.7870, AP to 0.7254, ECE increased to 0.1899, and Brier score to 0.2281. The frozen TN5000 policy assigned 83.7% to review, 1.0% to No-FNA, and 15.3% to FNA recommendation. No malignant image entered the No-FNA pathway, but FNA-recommendation PPV fell to 76.6%.
Conclusion: The framework showed strong internal discrimination and calibration, but limited external threshold transportability. Selective prediction may help identify images unsuitable for automated triage, but local recalibration, threshold validation, and prospective clinical evaluation are required before deployment.