Breaking Shortcut Learning for Cross-Trial EEG-Guided Target Speech Extraction via Two-Stage Training

2026-06-23

AI Summary

This work addresses the limited cross-trial generalization of existing end-to-end models in EEG-informed target speech extraction, which often exploit trial-specific EEG structures as shortcuts. To mitigate this shortcut learning, the authors propose TRUST-TSE, a two-stage training framework. In the first stage, contrastive pretraining with attention-based speaker negative sampling suppresses trial-identity cues; in the second stage, a confidence-weighted objective function based on EEG–speech alignment similarity refines the extraction performance. Evaluated under strict cross-trial protocols on the KUL and DTU datasets, the proposed method significantly outperforms current end-to-end baselines, demonstrating enhanced generalization and robustness.

Abstract

Recent end-to-end models for EEG-guided target speech extraction report impressive results, underscoring potential for neuro-steered hearing technologies. However, our analysis reveals that high within-trial performance can be driven by trial-specific EEG structure that acts as shortcuts for target selection, leading to poor generalization on unseen trials. To overcome this gap, we propose TRUST-TSE, a two-stage framework to mitigate shortcut learning. By introducing contrastive pretraining with attended-speaker negative sampling, we encourage the EEG encoder to capture fine-grained EEG--speech alignment while suppressing trial-identity cues. We also employ a confidence-weighted extraction objective based on EEG--source similarity to guide extraction using the learned representations. Experiments on KUL and DTU datasets show that TRUST-TSE outperforms end-to-end baselines under strict cross-trial protocols, addressing a key reliability bottleneck of existing approaches.

Problem

• shortcut learning
• cross-trial generalization
• EEG-guided speech extraction
• trial-specific bias
• neuro-steered hearing

Innovation

• shortcut learning
• EEG-guided speech extraction
• two-stage training
• contrastive pretraining
• cross-trial generalization