BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics

2025-05-26

AI Summary

In intracranial speech decoding from stereoelectroencephalography (sEEG) and electrocorticography (ECoG), task-relevant neural activity is sparsely distributed and highly entangled with task-irrelevant components. To address this, we propose the first coarse-to-fine two-stage neural disentanglement framework: (1) spatial-context-guided spatiotemporal modeling to identify functionally relevant neural ensembles; followed by (2) decoupled product quantization (DPQ) to isolate heterogeneous neural dynamics *within* these ensembles. Our method significantly enhances task-signal specificity and interpretability. Evaluated on two public sEEG and one epidural ECoG dataset, it consistently outperforms state-of-the-art speech decoders, achieving an average 12.6% improvement in decoding accuracy while yielding neurobiologically plausible, empirically verifiable mechanistic interpretations. The core innovation lies in the principled integration of functional localization and dynamical disentanglement—establishing a novel paradigm for invasive brain–computer interface–based speech decoding.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Speech
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Decoding speech directly from neural activity is a central goal in brain-computer interface (BCI) research. In recent years, exciting advances have been made through the growing use of intracranial field potential recordings, such as stereo-ElectroEncephaloGraphy (sEEG) and ElectroCorticoGraphy (ECoG). These neural signals capture rich population-level activity but present key challenges: (i) task-relevant neural signals are sparsely distributed across sEEG electrodes, and (ii) they are often entangled with task-irrelevant neural signals in both sEEG and ECoG. To address these challenges, we introduce a unified Coarse-to-Fine neural disentanglement framework, BrainStratify, which includes (i) identifying functional groups through spatial-context-guided temporal-spatial modeling, and (ii) disentangling distinct neural dynamics within the target functional group using Decoupled Product Quantization (DPQ). We evaluate BrainStratify on two open-source sEEG datasets and one (epidural) ECoG dataset, spanning tasks like vocal production and speech perception. Extensive experiments show that BrainStratify, as a unified framework for decoding speech from intracranial neural signals, significantly outperforms previous decoding methods. Overall, by combining data-driven stratification with neuroscience-inspired modularity, BrainStratify offers a robust and interpretable solution for speech decoding from intracranial recordings.

Problem

• Decoding speech from sparse intracranial neural signals
• Disentangling task-relevant and irrelevant neural dynamics
• Improving accuracy in brain-computer interface speech decoding

Innovation

• Coarse-to-Fine neural disentanglement framework
• Spatial-context-guided temporal-spatial modeling
• Decoupled Product Quantization (DPQ) technique