Hierarchic-EEG2Text: Assessing EEG-To-Text Decoding across Hierarchical Abstraction Levels

2026-02-24

AI Summary

This study addresses the challenge of fine-grained textual decoding from low signal-to-noise ratio electroencephalography (EEG) signals and investigates the sensitivity of neural representations to varying levels of semantic abstraction. To this end, we construct hierarchical classification tasks (episodes) based on WordNet and introduce a hierarchy-aware episode sampling strategy on the large-scale PEERS dataset, establishing the largest EEG episodic learning framework to date. We systematically evaluate multiple neural network architectures across multiple semantic granularities. Experimental results demonstrate that models significantly outperform on high-level abstract semantic categories compared to fine-grained ones, revealing for the first time the sensitivity of EEG signals to the depth of semantic abstraction. These findings offer novel insights for both brain–computer interface development and cognitive neuroscience research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

An electroencephalogram (EEG) records the spatially averaged electrical activity of neurons in the brain, measured from the human scalp. Prior studies have explored EEG-based classification of objects or concepts, often for passive viewing of briefly presented image or video stimuli, with limited classes. Because EEG exhibits a low signal-to-noise ratio, recognizing fine-grained representations across a large number of classes remains challenging; however, abstract-level object representations may exist. In this work, we investigate whether EEG captures object representations across multiple hierarchical levels, and propose episodic analysis, in which a Machine Learning (ML) model is evaluated across various, yet related, classification tasks (episodes). Unlike prior episodic EEG studies that rely on fixed or randomly sampled classes of equal cardinality, we adopt hierarchy-aware episode sampling using WordNet to generate episodes with variable classes of diverse hierarchy. We also present the largest episodic framework in the EEG domain for detecting observed text from EEG signals in the PEERS dataset, comprising $931538$ EEG samples under $1610$ object labels, acquired from $264$ human participants (subjects) performing controlled cognitive tasks, enabling the study of neural dynamics underlying perception, decision-making, and performance monitoring. We examine how the semantic abstraction level affects classification performance across multiple learning techniques and architectures, providing a comprehensive analysis. The models tend to improve performance when the classification categories are drawn from higher levels of the hierarchy, suggesting sensitivity to abstraction. Our work highlights abstraction depth as an underexplored dimension of EEG decoding and motivates future research in this direction.

Problem

• EEG-to-text decoding
• hierarchical abstraction
• semantic representation
• episodic analysis
• neural decoding

Innovation

• hierarchical EEG decoding
• episodic learning
• semantic abstraction
• EEG-to-text
• WordNet-based sampling