Is Uncertainty Quantification a Viable Alternative to Learned Deferral?

2025-08-04

AI Summary

This study addresses decision deferral under out-of-distribution (OOD) data—such as clinical deployment-induced distribution shifts—in AI–physician collaborative diagnosis, using glaucoma fundus image classification as a testbed. It systematically compares uncertainty quantification (UQ) methods (e.g., Monte Carlo Dropout, Deep Ensembles) against supervised learning-to-defer (L2D) approaches in terms of safety and robustness. Evaluated on a large-scale real-world fundus image dataset, UQ methods demonstrate superior OOD detection accuracy without requiring OOD labels, achieving significantly higher deferral precision and classification robustness under distribution shift. In contrast, L2D models—dependent on in-distribution labeled data for training—exhibit weaker generalization to unseen OOD scenarios. The results indicate that UQ serves as a lightweight, retraining-free, and reliable alternative for safe clinical AI deployment, offering a novel paradigm for trustworthy AI integration in healthcare.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Artificial Intelligence (AI) holds the potential to dramatically improve patient care. However, it is not infallible, necessitating human-AI-collaboration to ensure safe implementation. One aspect of AI safety is the models' ability to defer decisions to a human expert when they are likely to misclassify autonomously. Recent research has focused on methods that learn to defer by optimising a surrogate loss function that finds the optimal trade-off between predicting a class label or deferring. However, during clinical translation, models often face challenges such as data shift. Uncertainty quantification methods aim to estimate a model's confidence in its predictions. However, they may also be used as a deferral strategy which does not rely on learning from specific training distribution. We hypothesise that models developed to quantify uncertainty are more robust to out-of-distribution (OOD) input than learned deferral models that have been trained in a supervised fashion. To investigate this hypothesis, we constructed an extensive evaluation study on a large ophthalmology dataset, examining both learned deferral models and established uncertainty quantification methods, assessing their performance in- and out-of-distribution. Specifically, we evaluate their ability to accurately classify glaucoma from fundus images while deferring cases with a high likelihood of error. We find that uncertainty quantification methods may be a promising choice for AI deferral.

Problem

• Comparing uncertainty quantification vs learned deferral for AI decisions
• Assessing robustness to out-of-distribution data in medical AI models
• Evaluating glaucoma classification accuracy with error-prone case deferral

Innovation

• Uses uncertainty quantification for deferral decisions
• Compares learned deferral with uncertainty methods
• Tests robustness on out-of-distribution data