Modality-AGnostic Image Cascade (MAGIC) for Multi-Modality Cardiac Substructure Segmentation

2025-06-12

AI Summary

In radiotherapy planning, existing methods exhibit poor generalizability across multi-modal cardiac imaging (Sim-CT, MR-Linac, CCTA) and struggle to accurately differentiate overlapping anatomical structures—particularly coronary arteries versus major vessels and cardiac conduction nodes. Method: We propose the first modality-agnostic image-cascaded U-Net, built upon the nnU-Net backbone. It employs a replicated encoder-decoder architecture for end-to-end segmentation of 20 cardiac substructures and integrates a hybrid loss combining Dice coefficient with Wilcoxon rank-sum test regularization to enhance discrimination in ambiguous, overlapping regions. Results: Our model achieves mean Dice scores of 0.75 (Sim-CT), 0.68 (MR-Linac), and 0.80 (CCTA) on multi-modal test sets—outperforming state-of-the-art methods on 57% of evaluation metrics while reducing parameter count by 32%, thereby enabling real-time clinical deployment.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Cardiac substructures are essential in thoracic radiation therapy planning to minimize risk of radiation-induced heart disease. Deep learning (DL) offers efficient methods to reduce contouring burden but lacks generalizability across different modalities and overlapping structures. This work introduces and validates a Modality-AGnostic Image Cascade (MAGIC) for comprehensive and multi-modal cardiac substructure segmentation. MAGIC is implemented through replicated encoding and decoding branches of an nnU-Net-based, U-shaped backbone conserving the function of a single model. Twenty cardiac substructures (heart, chambers, great vessels (GVs), valves, coronary arteries (CAs), and conduction nodes) from simulation CT (Sim-CT), low-field MR-Linac, and cardiac CT angiography (CCTA) modalities were manually delineated and used to train (n=76), validate (n=15), and test (n=30) MAGIC. Twelve comparison models (four segmentation subgroups across three modalities) were equivalently trained. All methods were compared for training efficiency and against reference contours using the Dice Similarity Coefficient (DSC) and two-tailed Wilcoxon Signed-Rank test (threshold, p<0.05). Average DSC scores were 0.75(0.16) for Sim-CT, 0.68(0.21) for MR-Linac, and 0.80(0.16) for CCTA. MAGIC outperforms the comparison in 57% of cases, with limited statistical differences. MAGIC offers an effective and accurate segmentation solution that is lightweight and capable of segmenting multiple modalities and overlapping structures in a single model. MAGIC further enables clinical implementation by simplifying the computational requirements and offering unparalleled flexibility for clinical settings.

Problem

• Segment cardiac substructures across multiple imaging modalities
• Reduce contouring burden in thoracic radiation therapy planning
• Improve generalizability for overlapping structures in deep learning

Innovation

• Modality-AGnostic Image Cascade (MAGIC) for segmentation
• U-shaped nnU-Net backbone with replicated branches
• Lightweight single-model multi-modality segmentation