RaLMPH: Reliability-aware Learning for Multi-Pathologist Harmonization in Whole-Slide Image Classification

2026-06-13

AI Summary

This work addresses the challenge of substantial inter-pathologist annotation disagreement in whole-slide image (WSI) classification and the difficulty of effectively integrating locally inconsistent labels within multiple instance learning (MIL) frameworks. To this end, the authors propose a reliability-aware multi-expert annotation reconciliation framework. The method introduces, for the first time in MIL, a reliability field that combines local neighborhood structure with expert uncertainty (measured by entropy) to identify instance-level trustworthy reference neighborhoods. This enables localized annotator ranking and adaptive gated fusion of annotations. Experiments on a real-world clinical WSI dataset annotated by six pathologists, as well as on simulated benchmarks, demonstrate that the proposed approach significantly outperforms existing methods, yielding improved annotation reconciliation quality and enhanced downstream MIL performance.

Abstract

Multiple Instance Learning (MIL) is a standard paradigm for Whole-Slide Image (WSI) analysis and has achieved strong results in computational pathology. However, most MIL pipelines assume a single "gold" label per slide, which conflicts with clinical practice where substantial inter-pathologist variability is common. Existing multi-annotator learning and label-refinement methods typically estimate global annotator reliability or rely on single-instance assumptions, making them poorly suited to MIL and to localized diagnostic contexts where experts disagree. We propose RaLMPH (Reliability-aware Learning for Multi-Pathologist Harmonization), a MIL-based label reconciliation framework for WSIs annotated by multiple pathologists. RaLMPH introduces a reliability field that jointly models (i) local neighborhood structure in WSI feature space and (ii) expert uncertainty (entropy), enabling per-sample identification of trustworthy reference neighborhoods. Leveraging this field, RaLMPH performs sample-wise local annotator ranking to select reliable opinions per slide and applies an adaptive gating mechanism to fuse labels conditioned on local reliability. Experiments on a clinical WSI dataset with labels from six pathologists, as well as controlled simulated benchmarks, show that RaLMPH consistently outperforms existing approaches. Further analyses clarify how our reliability-aware mechanism improves label reconciliation and downstream MIL performance.

Problem

• Whole-Slide Image
• Multiple Instance Learning
• Inter-pathologist Variability
• Label Reconciliation
• Computational Pathology

Innovation

• Multiple Instance Learning
• Label Reconciliation
• Reliability-aware Learning
• Whole-Slide Image
• Inter-pathologist Variability