Meta-D: Metadata-Aware Architectures for Brain Tumor Analysis and Missing-Modality Segmentation

2026-03-05

AI Summary

This work addresses the performance degradation in brain tumor segmentation caused by missing MRI modalities by proposing a dynamic feature modulation approach that integrates metadata—such as MRI sequence type and scanning plane—into the feature extraction process. For the first time, scanning metadata is embedded as a conditioning signal to guide feature representation, and a metadata-driven cross-attention routing mechanism is designed to enable more robust information focusing under modality absence. By combining a 2D detection module with a 3D segmentation architecture, the method achieves notable gains in both efficiency and accuracy: the 2D detection F1-score improves by up to 2.62%, the 3D segmentation Dice score increases by as much as 5.12%, and the model parameter count is reduced by 24.1%.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We present Meta-D, an architecture that explicitly leverages categorical scanner metadata such as MRI sequence and plane orientation to guide feature extraction for brain tumor analysis. We aim to improve the performance of medical image deep learning pipelines by integrating explicit metadata to stabilize feature representations. We first evaluate this in 2D tumor detection, where injecting sequence (e.g., T1, T2) and plane (e.g., axial) metadata dynamically modulates convolutional features, yielding an absolute increase of up to 2.62% in F1-score over image-only baselines. Because metadata grounds feature extraction when data are available, we hypothesize it can serve as a robust anchor when data are missing. We apply this to 3D missing-modality tumor segmentation. Our Transformer Maximizer utilizes metadata-based cross-attention to isolate and route available modalities, ensuring the network focuses on valid slices. This targeted attention improves brain tumor segmentation Dice scores by up to 5.12% under extreme modality scarcity while reducing model parameters by 24.1%.

Problem

• brain tumor segmentation
• missing-modality
• metadata
• medical image analysis
• deep learning

Innovation

• metadata-aware architecture
• missing-modality segmentation
• cross-attention mechanism
• brain tumor analysis
• feature modulation