The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

2026-09-09

AI Summary

本文针对非侵入式语音解码中信号噪声比低的问题，提出了一种通过语义表示空间进行文本重构的新方法Brain2Semantics2Text，提高了句子级别的解码效果。

Abstract

Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.

Problem

• non-invasive speech decoding
• semantic representations
• neural recordings
• signal-to-noise ratio

Innovation

• Semantic Representations
• Non-Invasive Speech Decoding
• Brain2Semantics2Text
• Semantic Embedding