Assembling the Mind's Mosaic: Towards EEG Semantic Intent Decoding

2026-01-28

AI Summary

Current brain–computer interfaces for natural language communication are limited by oversimplified semantic representations and a lack of interpretability. This work proposes a Semantic Intent Decoding (SID) framework that maps neural activity to a set of composable, continuous, and scalable semantic units. To jointly decode multiple semantic units from both EEG and clinical stereo-EEG (SEEG) signals, the authors design BrainMosaic, a deep architecture that enables semantic-guided reconstruction of coherent sentences. By moving beyond conventional paradigms—either fixed-category classification or unconstrained generation—the proposed approach achieves significantly superior performance on multilingual EEG and SEEG datasets, enabling more expressive, interpretable, and high-fidelity neural communication of natural language.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Brain-Sensing and Analysis
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Enabling natural communication through brain-computer interfaces (BCIs) remains one of the most profound challenges in neuroscience and neurotechnology. While existing frameworks offer partial solutions, they are constrained by oversimplified semantic representations and a lack of interpretability. To overcome these limitations, we introduce Semantic Intent Decoding (SID), a novel framework that translates neural activity into natural language by modeling meaning as a flexible set of compositional semantic units. SID is built on three core principles: semantic compositionality, continuity and expandability of semantic space, and fidelity in reconstruction. We present BrainMosaic, a deep learning architecture implementing SID. BrainMosaic decodes multiple semantic units from EEG/SEEG signals using set matching and then reconstructs coherent sentences through semantic-guided reconstruction. This approach moves beyond traditional pipelines that rely on fixed-class classification or unconstrained generation, enabling a more interpretable and expressive communication paradigm. Extensive experiments on multilingual EEG and clinical SEEG datasets demonstrate that SID and BrainMosaic offer substantial advantages over existing frameworks, paving the way for natural and effective BCI-mediated communication.

Problem

• brain-computer interface
• semantic decoding
• natural language
• EEG
• interpretability

Innovation

• Semantic Intent Decoding
• BrainMosaic
• EEG/SEEG decoding
• compositional semantics
• brain-computer interface