Bridging the Gap in Missing Modalities: Leveraging Knowledge Distillation and Style Matching for Brain Tumor Segmentation

2025-07-30

AI Summary

To address degraded brain tumor segmentation accuracy, blurred boundaries, and insufficient cross-modal feature transfer caused by missing critical imaging modalities in multimodal medical images, this paper proposes a robust segmentation framework integrating multi-scale Transformer-based knowledge distillation, dual-modality logit distillation, and global style matching. Hierarchical knowledge transfer enhances structural consistency between teacher and student models, while style-matching constraints enforce cross-modal representation alignment and texture-invariant modeling. Evaluated on BraTS and FeTS 2024 benchmarks, the method significantly improves segmentation robustness: average Dice score increases by 2.1%, and HD95 decreases by 18.7%. Notably, it maintains high accuracy under severe modality absence—e.g., with single-modality input (T1- or T2-only)—demonstrating effective compensation for missing modalities.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Accurate and reliable brain tumor segmentation, particularly when dealing with missing modalities, remains a critical challenge in medical image analysis. Previous studies have not fully resolved the challenges of tumor boundary segmentation insensitivity and feature transfer in the absence of key imaging modalities. In this study, we introduce MST-KDNet, aimed at addressing these critical issues. Our model features Multi-Scale Transformer Knowledge Distillation to effectively capture attention weights at various resolutions, Dual-Mode Logit Distillation to improve the transfer of knowledge, and a Global Style Matching Module that integrates feature matching with adversarial learning. Comprehensive experiments conducted on the BraTS and FeTS 2024 datasets demonstrate that MST-KDNet surpasses current leading methods in both Dice and HD95 scores, particularly in conditions with substantial modality loss. Our approach shows exceptional robustness and generalization potential, making it a promising candidate for real-world clinical applications. Our source code is available at https://github.com/Quanato607/MST-KDNet.

Problem

• Addressing brain tumor segmentation with missing imaging modalities
• Improving tumor boundary segmentation and feature transfer issues
• Enhancing robustness in medical image analysis under modality loss

Innovation

• Multi-Scale Transformer Knowledge Distillation
• Dual-Mode Logit Distillation
• Global Style Matching Module