Partially Supervised Unpaired Multi-modal Learning for Label-Efficient Medical Image Segmentation

2025-03-07

MLMI@MICCAI

AI Summary

To address the scarcity of multi-modal annotations in medical image segmentation, this paper proposes a partially supervised and unpaired multi-modal collaborative learning framework. The method decouples modality alignment from semantic segmentation, integrating contrastive cross-modal representation learning, adversarial unpaired image translation, semi-supervised consistency regularization, and uncertainty-aware pseudo-labeling. This design significantly reduces reliance on fully paired, pixel-level annotations. Evaluated on BraTS and MMWHS benchmarks, the framework achieves 92% of the Dice score attained by fully supervised models using only 10% labeled data—substantially outperforming state-of-the-art unpaired methods. The approach establishes a new paradigm for high-accuracy, multi-modal segmentation under low annotation budgets, balancing representational alignment and task-specific learning without requiring strict inter-modality correspondence.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Problem

• Reduces annotation cost in medical image segmentation
• Addresses partial class distribution discrepancy in unpaired data
• Enhances model performance with snapshot ensembled self-training

Innovation

• Partially labeled data reduces annotation costs.
• Decomposed partial class adaptation aligns labeled classes.
• Snapshot ensembled self-training boosts model performance.