Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring

2026-07-15

AI Summary

This study addresses the limitations of existing second-language pronunciation assessment approaches, which predominantly focus on segmental features while underrepresenting suprasegmental dimensions such as rhythm and intonation, and often rely heavily on annotated data, limiting their applicability in low-resource settings. To overcome these challenges, this work proposes a text- and annotation-free multidimensional pronunciation evaluation framework. Leveraging self-supervised WavLM representations and dynamic time warping (DTW), the method quantifies rhythmic proficiency through the warping path distortion of DTW alignments and evaluates intonation by integrating prosodic residuals, fundamental frequency, and intensity features. Experimental results demonstrate that the proposed approach surpasses human inter-rater consistency in phoneme-level scoring, achieves near-human performance in rhythm assessment, and, despite modest effectiveness in intonation evaluation, validates the feasibility of an unsupervised pathway for comprehensive pronunciation assessment.

Abstract

L2 speech assessment has traditionally focused on phonetic assessment, leaving the scoring of suprasegmental features such as rhythm and intonation underexplored. Moreover, assessment methods often require training with labeled L2 speech data, making them difficult to apply in low-resource settings. We investigate whether DTW over self-supervised WavLM representations can provide a text-free framework for assessing phonetic accuracy, rhythm, and intonation in English and Japanese L2 speech. Results show that a basic DTW-based approach that compares learner speech to native templates exceeds human agreement on holistic and sentence-level phonetic scoring. For rhythm, we introduce methods that measure the degree of warping in the DTW alignment path; our best method approaches human-level performance. For intonation, we combine DTW distance over prosodic residuals with pitch and intensity features, but performance remains more modest on some tasks. Our results point to self-supervised representations as a promising, text-free basis for multi-aspect pronunciation assessment.

Problem

• L2 speech assessment
• suprasegmental features
• rhythm
• intonation
• low-resource settings

Innovation

• self-supervised learning
• dynamic time warping (DTW)
• suprasegmental assessment
• pronunciation scoring
• WavLM