Voxel-Level Brain States Prediction Using Swin Transformer

2025-06-13

AI Summary

This study addresses the challenge of predicting voxel-wise BOLD signal dynamics in human resting-state fMRI 7.2 seconds into the future, using only 23.04 seconds of raw time-series data. We propose the first whole-brain, voxel-level 4D Shifted Window Transformer (4D Swin Transformer), which directly models spatiotemporal dependencies across all four dimensions (x, y, z, t) without spatial downsampling, regional parcellation, or functional aggregation. The architecture comprises a 4D Swin encoder and a lightweight convolutional decoder, trained and validated on resting-state fMRI data (REST1) from 100 unrelated subjects in the Human Connectome Project. Results demonstrate that predicted signals faithfully reproduce ground-truth BOLD contrast and inter-voxel dynamic coupling patterns, achieving high-fidelity spatiotemporal reconstruction at the native 1.6 mm³ isotropic resolution. This work establishes a novel paradigm for accelerating fMRI acquisition, enabling real-time dynamic brain–computer interfaces, and probing intrinsic neurodynamics.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision
• Machine Learning: Time-Series/Data Streams

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Understanding brain dynamics is important for neuroscience and mental health. Functional magnetic resonance imaging (fMRI) enables the measurement of neural activities through blood-oxygen-level-dependent (BOLD) signals, which represent brain states. In this study, we aim to predict future human resting brain states with fMRI. Due to the 3D voxel-wise spatial organization and temporal dependencies of the fMRI data, we propose a novel architecture which employs a 4D Shifted Window (Swin) Transformer as encoder to efficiently learn spatio-temporal information and a convolutional decoder to enable brain state prediction at the same spatial and temporal resolution as the input fMRI data. We used 100 unrelated subjects from the Human Connectome Project (HCP) for model training and testing. Our novel model has shown high accuracy when predicting 7.2s resting-state brain activities based on the prior 23.04s fMRI time series. The predicted brain states highly resemble BOLD contrast and dynamics. This work shows promising evidence that the spatiotemporal organization of the human brain can be learned by a Swin Transformer model, at high resolution, which provides a potential for reducing the fMRI scan time and the development of brain-computer interfaces in the future.

Problem

• Predict future human resting brain states using fMRI
• Learn spatio-temporal fMRI data with Swin Transformer
• Reduce fMRI scan time and improve brain-computer interfaces

Innovation

• Uses 4D Swin Transformer for spatio-temporal learning
• Combines convolutional decoder for high-resolution prediction
• Predicts brain states from prior fMRI time series