A Foundation Model Framework for Multi-View MRI Classification of Extramural Vascular Invasion and Mesorectal Fascia Invasion in Rectal Cancer

2025-05-23

AI Summary

Subjective interpretation and poor cross-center generalizability hinder accurate assessment of extramural venous invasion (EVI) and mesorectal fascia invasion (MFI) in rectal cancer MRI. Method: We propose a multi-view MRI foundation model integrating frequency-domain scan standardization, dual-plane (axial/sagittal) feature collaboration, and an efficient frozen UMedPT representation coupled with lightweight logistic regression. Input consistency is enhanced via TotalSegmentator-based lesion cropping and self-supervised frequency harmonization. Results: On multi-center data, our method achieves AUCs of 0.82 for EVI detection (a 0.08 improvement over the challenge winner) and 0.77 for MFI classification. Frequency harmonization notably boosts MFI discrimination, while UMedPT_LR consistently outperforms CNN baselines (e.g., ResNet50, SeResNet) in F1-score and balanced accuracy. This framework significantly improves objectivity and clinical generalizability in risk stratification.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Background: Accurate MRI-based identification of extramural vascular invasion (EVI) and mesorectal fascia invasion (MFI) is pivotal for risk-stratified management of rectal cancer, yet visual assessment is subjective and vulnerable to inter-institutional variability. Purpose: To develop and externally evaluate a multicenter, foundation-model-driven framework that automatically classifies EVI and MFI on axial and sagittal T2-weighted MRI. Methods: This retrospective study used 331 pre-treatment rectal cancer MRI examinations from three European hospitals. After TotalSegmentator-guided rectal patch extraction, a self-supervised frequency-domain harmonization pipeline was trained to minimize scanner-related contrast shifts. Four classifiers were compared: ResNet50, SeResNet, the universal biomedical pretrained transformer (UMedPT) with a lightweight MLP head, and a logistic-regression variant using frozen UMedPT features (UMedPT_LR). Results: UMedPT_LR achieved the best EVI detection when axial and sagittal features were fused (AUC = 0.82; sensitivity = 0.75; F1 score = 0.73), surpassing the Chaimeleon Grand-Challenge winner (AUC = 0.74). The highest MFI performance was attained by UMedPT on axial harmonized images (AUC = 0.77), surpassing the Chaimeleon Grand-Challenge winner (AUC = 0.75). Frequency-domain harmonization improved MFI classification but variably affected EVI performance. Conventional CNNs (ResNet50, SeResNet) underperformed, especially in F1 score and balanced accuracy. Conclusion: These findings demonstrate that combining foundation model features, harmonization, and multi-view fusion significantly enhances diagnostic performance in rectal MRI.

Problem

• Automating MRI-based classification of EVI and MFI in rectal cancer
• Reducing inter-institutional variability in MRI assessment
• Improving diagnostic accuracy using multi-view fusion and harmonization

Innovation

• Self-supervised frequency-domain harmonization pipeline
• Multi-view fusion of axial and sagittal MRI
• UMedPT with lightweight MLP head