DECAF: Dynamic Envelope Context-Aware Fusion for Speech-Envelope Reconstruction from EEG

2026-02-22

AI Summary

This work addresses the limitations of existing EEG-based speech envelope reconstruction methods, which typically rely on static regression and neglect the continuous temporal dynamics of speech, resulting in low fidelity and high susceptibility to noise. To overcome this, the study formulates the task as a dynamic state estimation problem and introduces a state-space fusion framework. The proposed approach incorporates a context-aware module to extract recent speech priors and employs a learnable gating mechanism to adaptively integrate EEG neural signals with temporal contextual information. Experimental results on the ICASSP 2023 stimulus reconstruction benchmark demonstrate that the method significantly outperforms static EEG-only baselines, achieving notably improved accuracy and temporal coherence in reconstructed speech envelopes.

Technology Category

• Intelligent Robots: State Estimation
• Natural Language Processing: Speech
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Reconstructing the speech audio envelope from scalp neural recordings (EEG) is a central task for decoding a listener's attentional focus in applications like neuro-steered hearing aids. Current methods for this reconstruction, however, face challenges with fidelity and noise. Prevailing approaches treat it as a static regression problem, processing each EEG window in isolation and ignoring the rich temporal structure inherent in continuous speech. This study introduces a new, dynamic framework for envelope reconstruction that leverages this structure as a predictive temporal prior. We propose a state-space fusion model that combines direct neural estimates from EEG with predictions from recent speech context, using a learned gating mechanism to adaptively balance these cues. To validate this approach, we evaluate our model on the ICASSP 2023 Stimulus Reconstruction benchmark demonstrating significant improvements over static, EEG-only baselines. Our analyses reveal a powerful synergy between the neural and temporal information streams. Ultimately, this work reframes envelope reconstruction not as a simple mapping, but as a dynamic state-estimation problem, opening a new direction for developing more accurate and coherent neural decoding systems.

Problem

• speech-envelope reconstruction
• EEG
• temporal structure
• neural decoding
• dynamic fusion

Innovation

• dynamic fusion
• speech envelope reconstruction
• EEG decoding
• temporal context
• state-space model