Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates whether speaker-independent fine-tuning (SI-FT) on multi-speaker non-normative speech data can provide a better initialization for speaker-specific fine-tuning (SS-FT) compared to direct SS-FT from a generic pre-trained model. They propose a two-stage adaptation framework (SI-FT followed by SS-FT) and evaluate it on AphasiaBank and UA-Speech using Whisper-Large-v3 and Qwen3-ASR. Results demonstrate that the two-stage approach consistently improves personalization performance for non-normative speech recognition while maintaining acceptable out-of-domain performance on typical speech datasets.
Personalizing ASR for atypical speech gets a boost: pre-training on multi-speaker atypical data before speaker-specific fine-tuning significantly improves performance.
Personalizing automatic speech recognition (ASR) systems for non-normative speech, such as dysarthric and aphasic speech, is challenging. While speaker-specific fine-tuning (SS-FT) is widely used, it is typically initialized directly from a generic pre-trained model. Whether speaker-independent adaptation provides a stronger initialization prior under such mismatch remains unclear. In this work, we propose a two-stage adaptation framework consisting of speaker-independent fine-tuning (SI-FT) on multi-speaker non-normative data followed by SS-FT, and evaluate it through a controlled comparison with direct SS-FT under identical per-speaker conditions. Experiments on AphasiaBank and UA-Speech with Whisper-Large-v3 and Qwen3-ASR, alongside evaluation on typical-speech datasets TED-LIUM v3 and FLEURS, show that two-stage adaptation consistently improves personalization while maintaining manageable out-of-domain (OOD) trade-offs.