Escaping the BLEU Trap: A Signal-Grounded Framework with Decoupled Semantic Guidance for EEG-to-Text Decoding

2026-02-09

AI Summary

This work addresses key challenges in EEG-to-text decoding—namely semantic bias, insufficient utilization of neural signals, and the misleading nature of BLEU-based evaluation—by introducing SemKey, a multi-stage framework that guides text generation through four disentangled semantic objectives: sentiment, topic, length, and surprise. Semantic prompts serve as Queries while EEG embeddings act as Keys and Values injected into a large language model (LLM), compelling it to attend directly to neural inputs. The approach innovatively restructures the interaction between the encoder and the LLM, discarding generalization templates reliant on linguistic priors. For more rigorous assessment, the study introduces N-way retrieval accuracy and Fréchet distance to better capture semantic alignment and output diversity. Experiments demonstrate that SemKey significantly suppresses hallucination under noisy conditions and achieves state-of-the-art performance under robust evaluation protocols, markedly improving both semantic fidelity and lexical diversity.

Abstract

Decoding natural language from non-invasive EEG signals is a promising yet challenging task. However, current state-of-the-art models remain constrained by three fundamental limitations: Semantic Bias (mode collapse into generic templates), Signal Neglect (hallucination based on linguistic priors rather than neural inputs), and the BLEU Trap, where evaluation metrics are artificially inflated by high-frequency stopwords, masking a lack of true semantic fidelity. To address these challenges, we propose SemKey, a novel multi-stage framework that enforces signal-grounded generation through four decoupled semantic objectives: sentiment, topic, length, and surprisal. We redesign the interaction between the neural encoder and the Large Language Model (LLM) by injecting semantic prompts as Queries and EEG embeddings as Key-Value pairs, strictly forcing the model to attend to neural inputs. Furthermore, we move beyond standard translation metrics by adopting N-way Retrieval Accuracy and Fr\'echet Distance to rigorously assess diversity and alignment. Extensive experiments demonstrate that our approach effectively eliminates hallucinations on noise inputs and achieves SOTA performance on these robust protocols. Code will be released upon acceptance at https://github.com/xmed-lab/SemKey.

Problem

• EEG-to-Text Decoding
• Semantic Bias
• Signal Neglect
• BLEU Trap
• Neural Decoding

Innovation

• EEG-to-Text Decoding
• Signal-Grounded Generation
• Decoupled Semantic Guidance
• Large Language Model (LLM)
• Hallucination Mitigation