Uncertainty-Aware Retinal Vessel Segmentation via Ensemble Distillation

2025-09-15

AI Summary

Deep ensembles offer reliable uncertainty estimation for retinal vessel segmentation but incur prohibitive computational overhead, hindering clinical deployment. To address this, we propose an ensemble distillation framework that jointly transfers both predictive distributions and associated uncertainties from multiple teacher models into a single lightweight student network. Our method integrates probabilistic output distillation, temperature-scaled calibration, and segmentation performance constraints, enabling explicit uncertainty modeling while preserving pixel-level accuracy. Evaluated on DRIVE and FIVES, our approach reduces expected calibration error (ECE) by 32–45% compared to standard ensembles, achieves comparable segmentation performance (Dice, IoU), accelerates inference by 4.8×, and reduces parameter count by 91%. To the best of our knowledge, this is the first work to achieve high-fidelity uncertainty estimation and low computational cost simultaneously—significantly enhancing the clinical practicality of medical image segmentation models.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Uncertainty estimation is critical for reliable medical image segmentation, particularly in retinal vessel analysis, where accurate predictions are essential for diagnostic applications. Deep Ensembles, where multiple networks are trained individually, are widely used to improve medical image segmentation performance. However, training and testing costs increase with the number of ensembles. In this work, we propose Ensemble Distillation as a robust alternative to commonly used uncertainty estimation techniques by distilling the knowledge of multiple ensemble models into a single model. Through extensive experiments on the DRIVE and FIVES datasets, we demonstrate that Ensemble Distillation achieves comparable performance via calibration and segmentation metrics, while significantly reducing computational complexity. These findings suggest that Ensemble distillation provides an efficient and reliable approach for uncertainty estimation in the segmentation of the retinal vessels, making it a promising tool for medical imaging applications.

Problem

• Reducing computational cost of ensemble models for segmentation
• Improving uncertainty estimation in retinal vessel analysis
• Distilling multiple networks into a single efficient model

Innovation

• Ensemble Distillation for uncertainty estimation
• Distills multiple ensemble models into one
• Reduces computational complexity while maintaining performance