Confidence Score Guided Incremental and Speaker Adaptive Pseudo-Labeling for Semi-Supervised Elderly Speech Recognition

2026-06-15

AI Summary

This work addresses the challenges of low-quality pseudo-labels and speaker heterogeneity in semi-supervised elderly speech recognition by proposing a confidence-guided progressive pseudo-labeling approach integrated with learnable prompt-based speaker adaptive training. The method employs a curriculum learning strategy that progressively incorporates unlabeled data from high- to low-confidence samples, dynamically filtering reliable instances through a confidence estimation module. Evaluated on the DementiaBank Pitt and JCCOCC MoCA datasets, the proposed framework achieves significant performance gains, reducing word error rate (WER) by 1.45% (a relative improvement of 6.21%) and character error rate (CER) by 2.27% (a relative improvement of 6.98%). These results demonstrate enhanced robustness and generalization of the model for elderly speech recognition.

Abstract

This paper proposes a novel confidence score guided incremental and speaker adaptive pseudo-labeling approach for semi-supervised elderly speech recognition. It facilitates higher-quality pseudo-label selection and progressive refinement, while also mitigating speaker heterogeneity. A confidence estimation module is designed to rank the reliability of untranscribed data, enabling a curriculum learning trajectory that progressively folds in unlabeled data subsets from high to low confidence. Speaker-specific characteristics are captured through speaker adaptive training with learnable prompts. Experiments on the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets suggest that the proposed method outperforms the semi-supervised baseline using no confidence scores guided incremental or speaker adaptive pseudo-labeling by statistically significant word error rate (WER) or character error rate (CER) reductions of 1.45% and 2.27% absolute (6.21% and 6.98% relative).

Problem

• semi-supervised learning
• elderly speech recognition
• pseudo-labeling
• speaker heterogeneity
• confidence score

Innovation

• confidence score
• incremental pseudo-labeling
• speaker adaptive training
• semi-supervised speech recognition
• elderly speech