Multi-Teacher Distillation for Cross-Domain Streaming Electrolaryngeal Speech Encoding

2026-09-16

AI Summary

为解决电喉语音编码问题，提出多教师知识蒸馏框架训练轻量级流式内容编码器，通过自监督学习模型和微调识别模型提供目标，降低电喉语音错误率。

Abstract

Self-supervised learning (SSL) has improved speech representations, yet performance degrades in pathological domains such as electrolaryngeal (EL) speech, and the computational footprint of SSL models limits their applicability in real-time, on-device deployment. We propose a multi-teacher knowledge distillation framework to train a lightweight, streaming content encoder that generalizes across healthy (HE) and EL speech. Two teachers are distilled progressively: a frozen SSL model providing discrete phonetic cluster targets from HE speech, and an EL-fine-tuned speech recognition model supplying continuous bottleneck feature targets. Evaluated via downstream speech recognition, our approach reduces the EL word error rate to 21.2%, compared to 39.3% for the strongest zero-shot SSL baseline. Among causal convolutional, Transformer, Conformer, and Mamba-based student architectures, a Mel-Conformer achieves the best combination of EL accuracy and computational efficiency. The final encoder contains 21.9,M parameters and runs at a real-time factor of 0.30 under ONNX Runtime on a single CPU core.

Problem

• Self-supervised learning
• Electrolaryngeal speech
• Real-time deployment

Innovation

• multi-teacher distillation
• cross-domain generalization
• electrolaryngeal speech
• lightweight encoder
• real-time deployment