Adapting Foundation ASR Models to Dysarthric Speech: A Case Study

2026-06-30

AI Summary

This study addresses the significant performance degradation of general-purpose automatic speech recognition (ASR) systems on dysarthric speech, which hinders effective daily communication for affected individuals. The authors propose a personalized fine-tuning approach that leverages as little as 1.4 hours of user-specific read speech and mobile-device-collected correction feedback to fully fine-tune the Whisper base model, while also benchmarking alternatives such as LoRA adaptation and Qwen3-ASR. Experimental results demonstrate that the proposed method achieves a word error rate (WER) of 15.8% with only 1.4 hours of data, further improving to 9.7% when utilizing the full dataset of 92 hours of read speech and 8.8 hours of correction data. These results substantially outperform existing adaptation strategies, confirming the feasibility and deployment potential of highly effective personalized ASR under low-resource conditions.

Abstract

Automatic speech recognition (ASR) systems often perform poorly in dysarthric speech, limiting their usefulness to affected speakers in everyday communication. This paper presents a personalized ASR system for a dysarthric speaker, built by adapting a foundation ASR model to speaker-specific data. Using the TEQST tool, we collected 92 hours of read speech and later added 8.8 hours of user corrections gathered through a deployed mobile application. Starting from Whisper, fine-tuning reduced word error rate to 15.8% with only 1.4 hours of adaptation data, reached 10.7% with 22.5 hours, and achieved the best result of 9.7% when using all available data including the corrections. Using LoRA adaptation and/or Qwen3-ASR as foundation model performed worse in this setting. The results show that personalized fine-tuning can make foundation ASR models substantially more effective for dysarthric speech and suitable for practical deployment.

Problem

• dysarthric speech
• automatic speech recognition
• ASR adaptation
• speech impairment
• personalized ASR

Innovation

• personalized ASR
• dysarthric speech
• foundation model adaptation
• fine-tuning
• Whisper