Towards Scalable and Robust White Matter Lesion Localization via Multimodal Deep Learning

2025-06-27

AI Summary

This study addresses key challenges in white matter hyperintensity (WMH) segmentation and spatial localization—namely, missing modalities and insufficient integration of anatomical priors—by proposing a modality-interchangeable multi-task deep learning framework. Methodologically, it fuses FLAIR and T1-weighted MRI directly in native space, supporting flexible single- or multi-modal input; incorporates a modality dropout mechanism to enhance robustness against missing modalities; and jointly predicts WMH segmentation masks and anatomical region labels to enable lesion-anatomy co-modeling. Experiments on the MICCAI WMH Segmentation Challenge dataset demonstrate that multi-modal input significantly improves segmentation accuracy (Dice score +3.2%), while the modality-interchangeable design maintains stable performance even under single-modality conditions. Furthermore, anatomically informed predictions facilitate more precise differential diagnosis between cerebral small vessel disease and neurodegenerative disorders.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

White matter hyperintensities (WMH) are radiological markers of small vessel disease and neurodegeneration, whose accurate segmentation and spatial localization are crucial for diagnosis and monitoring. While multimodal MRI offers complementary contrasts for detecting and contextualizing WM lesions, existing approaches often lack flexibility in handling missing modalities and fail to integrate anatomical localization efficiently. We propose a deep learning framework for WM lesion segmentation and localization that operates directly in native space using single- and multi-modal MRI inputs. Our study evaluates four input configurations: FLAIR-only, T1-only, concatenated FLAIR and T1, and a modality-interchangeable setup. It further introduces a multi-task model for jointly predicting lesion and anatomical region masks to estimate region-wise lesion burden. Experiments conducted on the MICCAI WMH Segmentation Challenge dataset demonstrate that multimodal input significantly improves the segmentation performance, outperforming unimodal models. While the modality-interchangeable setting trades accuracy for robustness, it enables inference in cases with missing modalities. Joint lesion-region segmentation using multi-task learning was less effective than separate models, suggesting representational conflict between tasks. Our findings highlight the utility of multimodal fusion for accurate and robust WMH analysis, and the potential of joint modeling for integrated predictions.

Problem

• Accurate segmentation and localization of white matter hyperintensities (WMH)
• Handling missing modalities in multimodal MRI for WMH detection
• Integrating anatomical localization efficiently in WMH analysis

Innovation

• Multimodal deep learning for lesion segmentation
• Handles missing modalities with interchangeable setup
• Multi-task learning for lesion and region prediction