Segmentation-free Goodness of Pronunciation

2025-07-18

AI Summary

Existing pronunciation error detection methods rely on phoneme-level forced alignment, limiting both accuracy and direct applicability to CTC-based end-to-end ASR models. This work proposes two alignment-free pronunciation quality assessment methods: Self-Alignment GOP (GOP-SA) and Alignment-Free GOP (GOP-AF), the first approaches to theoretically define and realize alignment-free pronunciation evaluation. Our methods jointly leverage CTC-ASR’s implicit alignment, dynamic context aggregation, acoustic model sharpness compensation, and cross-model probability normalization to enhance robustness. Evaluated on CMU Kids and Speechocean762, GOP-AF achieves state-of-the-art performance in phoneme-level assessment, significantly outperforming prevailing alignment-dependent baselines.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Evaluation and Analysis
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Mispronunciation detection and diagnosis (MDD) is a significant part in modern computer aided language learning (CALL) systems. Within MDD, phoneme-level pronunciation assessment is key to helping L2 learners improve their pronunciation. However, most systems are based on a form of goodness of pronunciation (GOP) which requires pre-segmentation of speech into phonetic units. This limits the accuracy of these methods and the possibility to use modern CTC-based acoustic models for their evaluation. In this study, we first propose self-alignment GOP (GOP-SA) that enables the use of CTC-trained ASR models for MDD. Next, we define a more general alignment-free method that takes all possible alignments of the target phoneme into account (GOP-AF). We give a theoretical account of our definition of GOP-AF, an implementation that solves potential numerical issues as well as a proper normalization which makes the method applicable with acoustic models with different peakiness over time. We provide extensive experimental results on the CMU Kids and Speechocean762 datasets comparing the different definitions of our methods, estimating the dependency of GOP-AF on the peakiness of the acoustic models and on the amount of context around the target phoneme. Finally, we compare our methods with recent studies over the Speechocean762 data showing that the feature vectors derived from the proposed method achieve state-of-the-art results on phoneme-level pronunciation assessment.

Problem

• Eliminates need for speech pre-segmentation in pronunciation assessment
• Enables CTC-trained ASR models for mispronunciation detection
• Improves phoneme-level pronunciation evaluation accuracy

Innovation

• Self-alignment GOP enables CTC-trained ASR models
• Alignment-free method considers all phoneme alignments
• Normalization adapts to acoustic models' peakiness