MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data

2025-02-07

AI Summary

Cross-subject visual decoding from fMRI data remains challenging due to data scarcity and substantial inter-individual variability in functional brain organization. Method: We propose the first explicit brain functional alignment framework, introducing an interpretable Brain Transfer Matrix to map cortical functional representations across subjects. Our approach jointly optimizes a multi-level soft alignment loss and a stimulus-condition-aware contrastive loss, integrating linear mapping learning with transfer of pre-trained decoding models. Contribution/Results: On multiple public fMRI datasets, our method achieves significant improvements—12.6%–23.4% higher decoding accuracy—over state-of-the-art single-subject and cross-subject baselines, especially under limited-data regimes. Beyond performance gains, the framework provides neuroscientific interpretability by enabling principled modeling of cross-subject functional correspondences and generalization analysis, thereby overcoming fundamental limitations of conventional single-subject paradigms.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Brain decoding aims to reconstruct visual perception of human subject from fMRI signals, which is crucial for understanding brain's perception mechanisms. Existing methods are confined to the single-subject paradigm due to substantial brain variability, which leads to weak generalization across individuals and incurs high training costs, exacerbated by limited availability of fMRI data. To address these challenges, we propose MindAligner, an explicit functional alignment framework for cross-subject brain decoding from limited fMRI data. The proposed MindAligner enjoys several merits. First, we learn a Brain Transfer Matrix (BTM) that projects the brain signals of an arbitrary new subject to one of the known subjects, enabling seamless use of pre-trained decoding models. Second, to facilitate reliable BTM learning, a Brain Functional Alignment module is proposed to perform soft cross-subject brain alignment under different visual stimuli with a multi-level brain alignment loss, uncovering fine-grained functional correspondences with high interpretability. Experiments indicate that MindAligner not only outperforms existing methods in visual decoding under data-limited conditions, but also provides valuable neuroscience insights in cross-subject functional analysis. The code will be made publicly available.

Problem

• Cross-subject brain decoding
• Limited fMRI data
• Functional alignment framework

Innovation

• Brain Transfer Matrix projection
• Multi-level brain alignment loss
• Cross-subject functional analysis