Probabilistic Modeling of Multi-rater Medical Image Segmentation for Diversity and Personalization

2025-11-30

AI Summary

Medical image segmentation faces dual uncertainty arising from ambiguous object boundaries and inter-observer variability among experts, yet existing methods struggle to simultaneously capture segmentation diversity and expert-specific characteristics. To address this, we propose a dual latent-variable probabilistic framework that disentangles expert preference (personalization) from boundary ambiguity (diversity), jointly optimized via variational inference. Our approach integrates conditional generative modeling with deep probabilistic networks, enabling collaborative learning of individual annotation styles and structural uncertainty from multi-expert labels. Evaluated on the NPC and LIDC-IDRI datasets, our method significantly outperforms state-of-the-art approaches, achieving—for the first time—segmentation outputs that are both expert-specific and clinically plausible in their diversity. This work establishes a novel paradigm for uncertainty-aware, personalized computer-aided diagnosis.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Inference

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Medical image segmentation is inherently influenced by data uncertainty, arising from ambiguous boundaries in medical scans and inter-observer variability in diagnosis. To address this challenge, previous works formulated the multi-rater medical image segmentation task, where multiple experts provide separate annotations for each image. However, existing models are typically constrained to either generate diverse segmentation that lacks expert specificity or to produce personalized outputs that merely replicate individual annotators. We propose Probabilistic modeling of multi-rater medical image Segmentation (ProSeg) that simultaneously enables both diversification and personalization. Specifically, we introduce two latent variables to model expert annotation preferences and image boundary ambiguity. Their conditional probabilistic distributions are then obtained through variational inference, allowing segmentation outputs to be generated by sampling from these distributions. Extensive experiments on both the nasopharyngeal carcinoma dataset (NPC) and the lung nodule dataset (LIDC-IDRI) demonstrate that our ProSeg achieves a new state-of-the-art performance, providing segmentation results that are both diverse and expert-personalized. Code can be found in https://github.com/AI4MOL/ProSeg.

Problem

• Models diverse and personalized multi-rater medical image segmentation
• Addresses expert variability and image ambiguity via probabilistic modeling
• Generates segmentation by sampling from learned latent distributions

Innovation

• Models expert preferences and boundary ambiguity via latent variables
• Uses variational inference for conditional probabilistic distributions
• Generates diverse and personalized segmentations through distribution sampling