BrainMT: A Hybrid Mamba-Transformer Architecture for Modeling Long-Range Dependencies in Functional MRI Data

2025-06-27

AI Summary

Existing fMRI modeling approaches struggle to effectively capture long-range spatiotemporal dependencies, limiting brain phenotypic prediction performance. To address this, we propose a novel bidirectional Mamba-Transformer fusion architecture. It introduces a temporally prioritized scanning mechanism to efficiently model global temporal dynamics, while leveraging self-attention for deep spatial relationship modeling. Furthermore, we design a staged spatiotemporal modeling framework incorporating bidirectional context awareness and cross-modal feature interaction to substantially enhance fMRI representation learning. Evaluated on two large-scale datasets—UK Biobank and the Human Connectome Project (HCP)—our method achieves state-of-the-art performance on both sex classification and cognitive ability prediction tasks, outperforming existing models by significant margins. This work establishes a scalable, highly expressive new paradigm for large-scale fMRI analysis.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advances in deep learning have made it possible to predict phenotypic measures directly from functional magnetic resonance imaging (fMRI) brain volumes, sparking significant interest in the neuroimaging community. However, existing approaches, primarily based on convolutional neural networks or transformer architectures, often struggle to model the complex relationships inherent in fMRI data, limited by their inability to capture long-range spatial and temporal dependencies. To overcome these shortcomings, we introduce BrainMT, a novel hybrid framework designed to efficiently learn and integrate long-range spatiotemporal attributes in fMRI data. Our framework operates in two stages: (1) a bidirectional Mamba block with a temporal-first scanning mechanism to capture global temporal interactions in a computationally efficient manner; and (2) a transformer block leveraging self-attention to model global spatial relationships across the deep features processed by the Mamba block. Extensive experiments on two large-scale public datasets, UKBioBank and the Human Connectome Project, demonstrate that BrainMT achieves state-of-the-art performance on both classification (sex prediction) and regression (cognitive intelligence prediction) tasks, outperforming existing methods by a significant margin. Our code and implementation details will be made publicly available at this https://github.com/arunkumar-kannan/BrainMT-fMRI

Problem

• Model long-range dependencies in fMRI data efficiently
• Overcome limitations of CNNs and transformers in fMRI analysis
• Improve accuracy in classification and regression tasks for fMRI

Innovation

• Hybrid Mamba-Transformer for fMRI dependencies
• Bidirectional Mamba captures global temporal interactions
• Transformer models global spatial relationships efficiently