Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation

2026-06-18

AI Summary

This study addresses the challenges of automatic dysarthric speech recognition, particularly the scarcity of data and high pronunciation variability that lead to significant performance degradation across varying severity levels. Building upon the Wav2Vec2 end-to-end framework, the authors propose four tailored speech enhancement strategies—speech rate modification, fundamental frequency adjustment, formant shifting, and vocal tract length perturbation—and apply severity-specific fine-tuning for mild, moderate, and severe dysarthria. Experimental results demonstrate substantial improvements, achieving word error rates of 9.02%, 38.11%, and 55.15% for the three severity levels, respectively, corresponding to relative reductions of 30.02%, 16.64%, and 15.47%. These findings confirm the differential efficacy of the proposed enhancement techniques in accommodating distinct pathological speech characteristics.

Abstract

Dysarthric speech recognition is crucial for facilitating effective communication among individuals with dysarthria. However, accurately recognizing dysarthric speech poses significant challenges due to varying severity levels and limited data availability. In this paper, we explore data augmentation techniques for dysarthric automatic speech recognition (ASR) systems by fine-tuning the End-to-End pre-trained Wav2Vec2 model, with a specific focus on severity levels. To address the challenges of data scarcity and the need for extensive data in fine-tuning pre-trained ASR systems for dysarthric speech, we investigate four prominent data augmentation methods: Speaking-Rate Modification (SRM), Pitch Modification (PM), Formant Modification (FM), and vocal tract Length Perturbation (VTLP), tailored to different aspects of dysarthria. The study uses individually fine-tuned Wav2Vec2 models for each severity class as baseline systems. Additionally, we conducted severity-specific fine-tuning of the ASR model using augmented data. Results demonstrate distinct efficacy patterns for each augmentation technique across severity levels. The best WERs were achieved with SRM ($s$=0.8) for \textit{low} (9.02\%) and \textit{medium} (38.11\%) severities, and with PM ($τ$=0.8) for \textit{high} severity (55.15\%), reflecting relative improvements of 30.02\%, 16.64\%, and 15.47\%, respectively. These results confirm the effectiveness of the augmentation methods in improving dysarthric ASR performance.

Problem

• dysarthric speech recognition
• data scarcity
• severity levels
• automatic speech recognition
• end-to-end ASR

Innovation

• data augmentation
• dysarthric speech recognition
• Wav2Vec2
• severity-specific fine-tuning
• in-domain augmentation