Search papers, labs, and topics across Lattice.
This study addresses the challenges faced by automatic speech recognition (ASR) systems in accurately transcribing dysarthric speech by adapting a foundation ASR model, specifically Whisper, to a dysarthric speaker's unique vocal patterns. Through the collection of 92 hours of read speech and 8.8 hours of user corrections via a mobile app, the researchers fine-tuned the model, achieving a significant reduction in word error rate from 15.8% to 9.7%. The findings underscore the effectiveness of personalized adaptation in enhancing ASR performance for individuals with speech impairments, paving the way for more accessible communication technologies.
Personalized fine-tuning of ASR models can reduce word error rates for dysarthric speech to as low as 9.7%, transforming communication for affected individuals.
Automatic speech recognition (ASR) systems often perform poorly in dysarthric speech, limiting their usefulness to affected speakers in everyday communication. This paper presents a personalized ASR system for a dysarthric speaker, built by adapting a foundation ASR model to speaker-specific data. Using the TEQST tool, we collected 92 hours of read speech and later added 8.8 hours of user corrections gathered through a deployed mobile application. Starting from Whisper, fine-tuning reduced word error rate to 15.8% with only 1.4 hours of adaptation data, reached 10.7% with 22.5 hours, and achieved the best result of 9.7% when using all available data including the corrections. Using LoRA adaptation and/or Qwen3-ASR as foundation model performed worse in this setting. The results show that personalized fine-tuning can make foundation ASR models substantially more effective for dysarthric speech and suitable for practical deployment.
CMU ML