NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction

2026-06-22

AI Summary

Reconstructing continuous speech from scalp electroencephalography (EEG) faces fundamental challenges due to weak signal strength, spatial smearing, substantial inter-subject variability, and the complex acoustic structure of speech. This work proposes NeuroSonic, the first approach to introduce deterministic conditional probability flow matching to the EEG-to-speech task. By embedding EEG and audio into a shared token space and employing a time-conditioned gated Transformer to parameterize the transport ordinary differential equation, NeuroSonic explicitly models the evolution of acoustic trajectories, enabling end-to-end deterministic speech synthesis. Evaluated across subjects on the CineBrain and EAV datasets, NeuroSonic significantly outperforms GAN-, diffusion-, and mean-flow-based baselines, achieving up to a 26.3% improvement in overall perceptual quality, with particularly robust performance on segments containing high levels of artifacts.

Abstract

Reconstructing continuous speech from scalp electroencephalography (EEG) remains fundamentally challenging. EEG provides a weak, spatially diffuse, and highly variable measurement of distributed cortical activity, whereas speech is organized as a coherent acoustic trajectory with strong harmonic and temporal structure. The resulting mismatch makes waveform regression unstable and causes stochastic multi-step generation to be sensitive to artifact-dependent conditioning and subject variability. We introduce NeuroSonic, a conditional flow-matching framework for EEG-to-speech reconstruction. Instead of predicting waveforms directly or refining them through stochastic denoising, NeuroSonic learns a deterministic probability-flow velocity field that transports a noise-corrupted acoustic state toward clean speech under EEG conditioning. EEG and audio are embedded into a shared token space and processed by a time-conditioned gated Transformer that parameterizes the transport ordinary differential equation. This formulation models trajectory evolution explicitly while avoiding iterative stochastic sampling. We evaluate NeuroSonic on the CineBrain and EAV benchmarks under cross-subject evaluation. Across both datasets, the proposed method improves distributional realism, spectral fidelity, and perceptual quality over representative GAN-, diffusion-, and mean-flow baselines, with up to a 26.3\% gain in overall perceptual quality. The performance gap is most evident in artifact-heavy segments, where conditioning variability is strongest. These findings indicate that deterministic conditional transport provides a stable and effective formulation for EEG-driven speech reconstruction. Code is available at https://github.com/Y-Research-SBU/NeuroSonic/ .

Problem

• EEG-to-speech reconstruction
• conditional generation
• speech synthesis
• brain-computer interface
• neural decoding

Innovation

• Conditional Flow Matching
• EEG-to-Speech Reconstruction
• Deterministic Transport
• Gated Transformer
• Probability Flow ODE