Large-Scale Pre-training Enables Multimodal AI Differentiation of Radiation Necrosis from Brain Metastasis Progression on Routine MRI

2025-11-22

AI Summary

Differentiating radiation necrosis (RN) from tumor progression after stereotactic radiosurgery (SRS) for brain metastases remains clinically challenging due to the invasiveness of biopsy—the diagnostic gold standard—and scarcity of labeled data. To address this, we propose a two-stage multimodal self-supervised learning framework based on Vision Transformer (ViT), jointly leveraging contrast-enhanced T1-weighted MRI and lesion segmentation masks for pretraining on large-scale unlabeled data, followed by few-shot fine-tuning for binary classification. Our approach innovatively integrates self-supervised pretraining, multimodal input fusion, and attention map visualization, substantially enhancing few-shot generalization and model interpretability. Evaluated on internal and external test sets, the method achieves AUCs of 0.947 and 0.821, respectively—outperforming fully supervised models and conventional radiomics approaches. These results demonstrate strong clinical feasibility and translational potential for noninvasive differential diagnosis in neuro-oncology.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Neuro-Symbolic Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

Background: Differentiating radiation necrosis (RN) from tumor progression after stereotactic radiosurgery (SRS) remains a critical challenge in brain metastases. While histopathology represents the gold standard, its invasiveness limits feasibility. Conventional supervised deep learning approaches are constrained by scarce biopsy-confirmed training data. Self-supervised learning (SSL) overcomes this by leveraging the growing availability of large-scale unlabeled brain metastases imaging datasets. Methods: In a two-phase deep learning strategy inspired by the foundation model paradigm, a Vision Transformer (ViT) was pre-trained via SSL on 10,167 unlabeled multi-source T1CE MRI sub-volumes. The pre-trained ViT was then fine-tuned for RN classification using a two-channel input (T1CE MRI and segmentation masks) on the public MOLAB dataset (n=109) using 20% of datasets as same-center held-out test set. External validation was performed on a second-center test cohort (n=28). Results: The self-supervised model achieved an AUC of 0.916 on the same-center test set and 0.764 on the second center test set, surpassing the fully supervised ViT (AUC 0.624/0.496; p=0.001/0.008) and radiomics (AUC 0.807/0.691; p=0.005/0.014). Multimodal integration further improved performance (AUC 0.947/0.821; p=0.073/0.001). Attention map visualizations enabled interpretability showing the model focused on clinically relevant lesion subregions. Conclusion: Large-scale pre-training on increasingly available unlabeled brain metastases datasets substantially improves AI model performance. A two-phase multimodal deep learning strategy achieved high accuracy in differentiating radiation necrosis from tumor progression using only routine T1CE MRI and standard clinical data, providing an interpretable, clinically accessible solution that warrants further validation.

Problem

• Differentiating radiation necrosis from brain metastasis progression using MRI
• Overcoming limited biopsy data with self-supervised learning on unlabeled datasets
• Developing interpretable AI for clinical differentiation of post-treatment brain lesions

Innovation

• Self-supervised pre-training on large unlabeled MRI datasets
• Vision Transformer fine-tuned with multimodal two-channel inputs
• Interpretable AI using attention maps for lesion analysis