Delving Aleatoric Uncertainty in Medical Image Segmentation via Vision Foundation Models

2026-04-12

AI Summary

Medical image segmentation is often hindered by data uncertainty arising from acquisition noise and ambiguous annotations, which compromises model robustness. This work presents the first systematic exploration of aleatoric uncertainty in medical imaging and introduces a semantic-aware scale derived from vision foundation models: it quantifies class-level uncertainty through the singular value energy of decoded features to assess sample difficulty. Building on this measure, two novel strategies are proposed—uncertainty-driven data filtering and dynamic class-wise loss weighting—combined with a label denoising mechanism to refine training. Evaluated across five public datasets encompassing both CT and MRI modalities, the approach consistently enhances the performance of multiple state-of-the-art segmentation networks, demonstrating the effectiveness and generalizability of explicit uncertainty modeling in medical image understanding.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Medical image segmentation supports clinical workflows by precisely delineating anatomical structures and lesions. However, medical image datasets medical image datasets suffer from acquisition noise and annotation ambiguity, causing pervasive data uncertainty that substantially undermines model robustness. Existing research focuses primarily on model architectural improvements and predictive reliability estimation, while systematic exploration of the intrinsic data uncertainty remains insufficient. To address this gap, this work proposes leveraging the universal representation capabilities of visual foundation models to estimate inherent data uncertainty. Specifically, we analyze the feature diversity of the model's decoded representations and quantify their singular value energy to define the semantic perception scale for each class, thereby measuring sample difficulty and aleatoric uncertainty. Based on this foundation, we design two uncertainty-driven application strategies: (1) the aleatoric uncertainty-aware data filtering mechanism to eliminate potentially noisy samples and enhance model learning quality; (2) the dynamic uncertainty-aware optimization strategy that adaptively adjusts class-specific loss weights during training based on the semantic perception scale, combined with a label denoising mechanism to improve training stability. Experimental results on five public datasets encompassing CT and MRI modalities and involving multi-organ and tumor segmentation tasks demonstrate that our method achieves significant and robust performance improvements across various mainstream network architectures, revealing the broad application potential of aleatoric uncertainty in medical image understanding and segmentation tasks.

Problem

• aleatoric uncertainty
• medical image segmentation
• data uncertainty
• annotation ambiguity
• acquisition noise

Innovation

• aleatoric uncertainty
• vision foundation models
• medical image segmentation
• semantic perception scale
• uncertainty-aware optimization