BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activity

2025-12-12

AI Summary

This study addresses the challenges of spatial information encoding and brain network pattern learning in multi-regional intracranial electroencephalography (iEEG). We propose a spatiotemporal-aware multi-scale Transformer model. Methodologically, we introduce three novel components: (1) a configurable spatial tokenization scheme—from channel-level to region-level granularity; (2) hierarchical positional encoding; and (3) masked latent reconstruction (MLR), a self-supervised pretraining task jointly optimizing region-level representation learning and channel-level signal reconstruction. Our key contribution is the empirical demonstration that region-level encoding substantially enhances downstream decoding performance: on multi-center iEEG datasets, average accuracy for motor intention and speech decoding improves by 12.3%, while channel-level reconstruction error (MAE) remains below 0.85—validating the effectiveness and practicality of cross-scale modeling.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Intracranial recordings have opened a unique opportunity to simultaneously measure activity across multiregional networks in the human brain. Recent works have focused on developing transformer-based neurofoundation models of such recordings that can generalize across subjects and datasets. However, these recordings exhibit highly complex spatiotemporal interactions across diverse spatial scales, from the single-channel scale to the scale of brain regions. As such, there remain critical open questions regarding how best to encode spatial information and how to design self-supervision tasks that enable the learning of brain network patterns and enhance downstream decoding performance using such high-dimensional, multiregional recordings. To allow for exploring these questions, we propose a new spatiotemporal transformer model of multiregional neural activity and a corresponding self-supervised masked latent reconstruction task, designed to enable flexibility in the spatial scale used for token encoding and masking. Applying this model on publicly available multiregional intracranial electrophysiology (iEEG) data, we demonstrate that adjusting the spatial scale for both token encoding and masked reconstruction significantly impacts downstream decoding. Further, we find that spatial encoding at larger scales than channel-level encoding, which is commonly used in existing iEEG transformer models, improves downstream decoding performance. Finally, we demonstrate that our method allows for region-level token encoding while also maintaining accurate channel-level neural reconstruction. Taken together, our modeling framework enables exploration of the spatial scales used for token encoding and masking, reveals their importance towards self-supervised pretraining of neurofoundation models of multiregional human brain activity, and enhances downstream decoding performance.

Problem

• Explores optimal spatial scale encoding in intracranial neural activity models
• Designs self-supervised tasks for learning brain network patterns
• Enhances downstream decoding performance using multiregional recordings

Innovation

• Transformer model with flexible spatial scale token encoding
• Self-supervised masked latent reconstruction task for brain networks
• Region-level encoding improves decoding over channel-level methods