Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal

2026-06-18

AI Summary

This work addresses the challenge of deploying lightweight pronunciation assessment systems without relying on costly non-native annotated data. The authors propose a fully native-data-driven approach that leverages self-supervised speech encoders combined with K-means clustering to generate discrete speech tokens. Pronunciation quality is quantified via surprisal scores derived from a token-based language model. To align reference text with these discrete units and extract error-sensitive features, they introduce a Text2DUnit–DTW module. Evaluated on SpeechOcean762, the method achieves a Pearson correlation coefficient (PCC) of 0.66—improving by 6 points over prior unsupervised methods and approaching supervised baselines—while demonstrating robust cross-dataset generalization on L2-ARCTIC. This approach significantly reduces reliance on labeled non-native speech while maintaining high assessment accuracy.

Abstract

Training automated pronunciation assessment often relies on labeled learner errors or non-native corpora that are costly to collect. We propose a lightweight framework trained only on native speech resources, operating unsupervised or lightly calibrated with a small set of scored utterances. At inference, learner speech is discretized with an SSL encoder and a K-means codebook. A token language model trained on native sequences computes surprisal where higher surprisal indicates phonotactic deviation. We add a transcript-guided Text2DUnit--DTW module that predicts native token sequences from reference text and aligns them to acoustic tokens to derive error-sensitive features. Surprisal and alignment features are fused via simple regression. On SpeechOcean762, PCC improves from 0.60 to 0.66 with transcript guidance, near supervised baselines. Cross-dataset evaluation on L2-ARCTIC shows consistent gains.

Problem

• pronunciation assessment
• discrete speech tokens
• surprisal
• non-native speech
• lightweight framework

Innovation

• discrete speech token
• surprisal
• pronunciation assessment
• SSL encoder
• Text2DUnit–DTW