Population Transformer: Learning Population-Level Representations of Neural Activity

2024-06-05

arXiv.org

AI Summary

To address the challenges of population-level neural representation learning and generalization—stemming from sparse, irregular, cross-subject and cross-device neural data with inconsistent electrode distributions—this paper introduces Population Transformer (PopT), the first self-supervised representation learning framework specifically designed for neural populations. PopT integrates spatiotemporal self-supervised pretraining, dynamic multi-channel sparse feature aggregation, and cross-modal time-series embedding alignment to construct transferable, population-level neural representations. Its key contributions include: (1) zero-shot cross-subject decoding capability; (2) substantial reduction in downstream annotation requirements (average 62% fewer labeled samples) and improved decoding accuracy (+8.3%); and (3) lightweight architecture, high interpretability, and strong generalization across subjects and devices. The code and pretrained models are publicly released to advance the practical deployment of invasive brain–computer interfaces.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We present a self-supervised framework that learns population-level codes for arbitrary ensembles of neural recordings at scal. We address two key challenges in scaling models with neural time-series data: sparse and variable electrode distribution across subjects and datasets. The Population Transformer (PopT) stacks on top of pretrained representations and enhances downstream decoding by enabling learned aggregation of multiple spatially-sparse data channels. The pretrained PopT lowers the amount of data required for downstream decoding experiments, while increasing accuracy, even on held-out subjects and tasks. Compared to end-to-end methods, this approach is computationally lightweight and more interpretable, while still retaining competitive performance. We further show how our framework is generalizable to multiple time-series embeddings and neural data modalities. Beyond decoding, we interpret the pretrained PopT and fine-tuned models to show how they can be used to extract neuroscience insights from massive amounts of data. We release our code as well as a pretrained PopT to enable off-the-shelf improvements in multi-channel intracranial data decoding and interpretability.

Problem

• Learns population-level codes for neural recordings at scale
• Addresses sparse and variable electrode distribution challenges
• Enhances decoding accuracy with less data and computational cost

Innovation

• Self-supervised framework for neural activity representation
• Learned aggregation of spatially-sparse neural data channels
• Generalizable to multiple time-series embeddings and modalities