MO-CTranS: A unified multi-organ segmentation model learning from multiple heterogeneously labelled datasets

2025-03-28

AI Summary

Clinical multi-organ segmentation faces practical challenges including decentralized multi-center data, inconsistent annotations, and partial label absence, limiting data utilization and generalizability of single-dataset models. To address this, we propose the first unified segmentation framework for heterogeneous partially labeled abdominal MRI (axial and coronal views, covering liver, kidneys, spleen, etc.). Our method features: (1) a task-specific token mechanism that explicitly models organ-level label conflicts; and (2) a CNN-Transformer hybrid architecture with multi-resolution feature fusion, jointly capturing local anatomical details and global contextual dependencies to enable robust collaborative training under partial labeling. Evaluated on multi-center datasets, our approach significantly outperforms mainstream baselines and state-of-the-art methods (p < 0.05). Crucially, the unified model achieves superior performance compared to individual dataset-specific models, demonstrating enhanced cross-site generalization and efficient knowledge transfer across heterogeneous annotation protocols.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Security and Privacy: Data transparency and provenance

Abstract

Multi-organ segmentation holds paramount significance in many clinical tasks. In practice, compared to large fully annotated datasets, multiple small datasets are often more accessible and organs are not labelled consistently. Normally, an individual model is trained for each of these datasets, which is not an effective way of using data for model learning. It remains challenging to train a single model that can robustly learn from several partially labelled datasets due to label conflict and data imbalance problems. We propose MO-CTranS: a single model that can overcome such problems. MO-CTranS contains a CNN-based encoder and a Transformer-based decoder, which are connected in a multi-resolution manner. Task-specific tokens are introduced in the decoder to help differentiate label discrepancies. Our method was evaluated and compared to several baseline models and state-of-the-art (SOTA) solutions on abdominal MRI datasets that were acquired in different views (i.e. axial and coronal) and annotated for different organs (i.e. liver, kidney, spleen). Our method achieved better performance (most were statistically significant) than the compared methods. Github link: https://github.com/naisops/MO-CTranS.

Problem

• Develops a unified model for multi-organ segmentation from heterogeneous datasets
• Addresses label conflicts and data imbalance in partially labelled datasets
• Improves segmentation accuracy across different imaging views and organs

Innovation

• CNN-Transformer hybrid model for segmentation
• Task-specific tokens handle label discrepancies
• Multi-resolution connection enhances learning