Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR

2026-07-06

AI Summary

This work addresses the performance bottleneck in Mandarin–English code-switching automatic speech recognition (ASR) caused by the scarcity of labeled data by proposing an innovative iterative pseudo-labeling semi-supervised training framework. For the first time in code-switching ASR, this approach integrates iterative pseudo-labeling to generate high-quality pseudo-labels for large-scale unlabeled data, combined with bilingual pretraining, a two-stage fine-tuning strategy, and multi-round iterative optimization. This significantly enhances the model’s ability to capture complex code-switching patterns. Evaluated on the SEAME dataset, the proposed method achieves relative reductions of 6.35% and 8.29% in mixed error rate on the devman and devsge subsets, respectively, demonstrating its effectiveness and state-of-the-art performance.

Abstract

Code-switching (CS), alternating languages within the same utterance, poses significant challenges for automatic speech recognition (ASR) due to limited CS training data. This paper applies an iterative pseudo-labeling training approach to CS-ASR for the first time, demonstrating its effectiveness in leveraging unlabeled data to improve CS-ASR performance. The approach comprises three phases: pseudo-label generation, two-stage bilingual model training, and iterative improvements. It begins by generating pseudo-labels from a large unlabeled corpus, creating a semi-supervised dataset. This dataset supports a two-stage training framework where the model is pre-trained and then fine-tuned on supervised CS data. Iterative refinements further enhance the model's accuracy in handling complex CS scenarios. Our approach significantly advances CS-ASR systems, achieving notable Mix Error Rate (MER) reductions on SEAME's devman (6.35%) and devsge (8.29%) subsets.

Problem

• Code-Switching
• Automatic Speech Recognition
• Mandarin-English
• Limited Training Data
• Pseudo-Labeling

Innovation

• iterative pseudo-labeling
• code-switching ASR
• semi-supervised learning
• bilingual model training
• Mix Error Rate