ALIGN: Adversarial Learning for Generalizable Speech Neuroprosthesis

2026-03-18

AI Summary

This study addresses the performance degradation in cross-session speech decoding for brain–computer interfaces caused by the non-stationarity of neural signals. To tackle this challenge, the authors propose ALIGN, a novel framework that introduces adversarial domain alignment into intracranial speech neuroprosthetics for the first time. ALIGN employs a multi-domain adversarial neural network to jointly optimize a feature encoder, a phoneme classifier, and a domain classifier in unlabeled new sessions, thereby learning session-invariant representations that effectively suppress session-specific interference. Experimental results demonstrate that ALIGN significantly reduces both phoneme and word error rates compared to existing baselines, achieving strong generalization through semi-supervised cross-session adaptation.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Speech

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Intracortical brain-computer interfaces (BCIs) can decode speech from neural activity with high accuracy when trained on data pooled across recording sessions. In realistic deployment, however, models must generalize to new sessions without labeled data, and performance often degrades due to cross-session nonstationarities (e.g., electrode shifts, neural turnover, and changes in user strategy). In this paper, we propose ALIGN, a session-invariant learning framework based on multi-domain adversarial neural networks for semi-supervised cross-session adaptation. ALIGN trains a feature encoder jointly with a phoneme classifier and a domain classifier operating on the latent representation. Through adversarial optimization, the encoder is encouraged to preserve task-relevant information while suppressing session-specific cues. We evaluate ALIGN on intracortical speech decoding and find that it generalizes consistently better to previously unseen sessions, improving both phoneme error rate and word error rate relative to baselines. These results indicate that adversarial domain alignment is an effective approach for mitigating session-level distribution shift and enabling robust longitudinal BCI decoding.

Problem

• cross-session generalization
• nonstationarity
• intracortical BCI
• speech neuroprosthesis
• domain shift

Innovation

• adversarial learning
• domain adaptation
• intracortical BCI
• session-invariant representation
• speech neuroprosthesis