From Flat to Round: Redefining Brain Decoding with Surface-Based fMRI and Cortex Structure

2025-07-22

AI Summary

Current fMRI-based visual reconstruction methods often neglect the brain’s spatial topology and inter-subject anatomical variability, resulting in low reconstruction fidelity, poor generalizability across subjects, and weak neuroscientific interpretability. To address these limitations, we propose a structure-aware spherical modeling framework: cortical fMRI signals are projected onto a spherical manifold to establish a continuous 2D representation; an individualized structural MRI (sMRI)-guided encoding module explicitly models anatomical variation; and a spherical tokenizer coupled with spherical convolutional networks—augmented by positive-sample mixup—is introduced to enhance data efficiency. Evaluated on multiple benchmarks, our method substantially outperforms state-of-the-art approaches, achieving significant improvements in reconstruction fidelity, cross-subject generalization, and neurobiological interpretability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Learning with Manifolds
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Reconstructing visual stimuli from human brain activity (e.g., fMRI) bridges neuroscience and computer vision by decoding neural representations. However, existing methods often overlook critical brain structure-function relationships, flattening spatial information and neglecting individual anatomical variations. To address these issues, we propose (1) a novel sphere tokenizer that explicitly models fMRI signals as spatially coherent 2D spherical data on the cortical surface; (2) integration of structural MRI (sMRI) data, enabling personalized encoding of individual anatomical variations; and (3) a positive-sample mixup strategy for efficiently leveraging multiple fMRI scans associated with the same visual stimulus. Collectively, these innovations enhance reconstruction accuracy, biological interpretability, and generalizability across individuals. Experiments demonstrate superior reconstruction performance compared to SOTA methods, highlighting the effectiveness and interpretability of our biologically informed approach.

Problem

• Reconstructing visual stimuli from fMRI brain activity data
• Overcoming flattened spatial information in brain decoding methods
• Addressing individual anatomical variations in fMRI analysis

Innovation

• Sphere tokenizer models fMRI as 2D spherical data
• Integrates sMRI for personalized anatomical encoding
• Uses positive-sample mixup for efficient fMRI scan utilization