Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG

2025-05-31

AI Summary

This work addresses the challenge of high-fidelity semantic decoding of continuous speech from intracranial electroencephalography (iEEG). We propose a two-stage transfer learning framework: first, a lightweight LSTM adapter aligns iEEG features to the semantic embedding space of a pre-trained language model (e.g., BERT); second, an end-to-end correction module generates fluent, unconstrained continuous text. Trained on only 30 minutes of neural data, our method significantly outperforms existing state-of-the-art approaches in low-resource settings, achieving high-accuracy semantic reconstruction. Its core innovations lie in (1) a novel semantic-space alignment mechanism that bridges neural dynamics and linguistic representations, and (2) a task-adapted, parameter-efficient architecture. The framework enables scalable, clinically viable real-time neural decoding—advancing practical brain–computer interfaces for speech restoration.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Decoding continuous language from neural signals remains a significant challenge in the intersection of neuroscience and artificial intelligence. We introduce Neuro2Semantic, a novel framework that reconstructs the semantic content of perceived speech from intracranial EEG (iEEG) recordings. Our approach consists of two phases: first, an LSTM-based adapter aligns neural signals with pre-trained text embeddings; second, a corrector module generates continuous, natural text directly from these aligned embeddings. This flexible method overcomes the limitations of previous decoding approaches and enables unconstrained text generation. Neuro2Semantic achieves strong performance with as little as 30 minutes of neural data, outperforming a recent state-of-the-art method in low-data settings. These results highlight the potential for practical applications in brain-computer interfaces and neural decoding technologies.

Problem

• Decoding continuous language from neural signals
• Reconstructing semantic content from intracranial EEG
• Overcoming limitations in unconstrained text generation

Innovation

• LSTM-based adapter aligns neural signals
• Corrector module generates natural text
• Works well with minimal neural data