Search papers, labs, and topics across Lattice.
This paper introduces a cross-utterance audio-textual prompts-based approach for zero-shot online speaker adaptation aimed at improving elderly speech recognition. By extracting and fusing speech and text embeddings from current and preceding utterances, the method generates compact speaker prompts that significantly enhance recognition accuracy compared to traditional i/x-vectors and ECAPA-TDNN features. Experiments demonstrate that this approach achieves statistically significant reductions in word and character error rates, alongside impressive real-time factor speed-ups, highlighting its effectiveness in practical applications.
Zero-shot adaptation can reduce word error rates by over 0.6% while achieving nearly 10 times faster real-time processing for elderly speech recognition.
This paper proposes a novel cross-utterance audio-textual prompts based speaker adaptation approach for elderly speech recognition. It enables zero-shot, real-time adaptation to unseen speakers. Speech and text embeddings are extracted from the current and a few preceding utterances, before being fused in a cross-modal manner to produce compact speaker prompts that are more consistent than i/x-vectors and ECAPA-TDNN features. Experiments on the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets suggest that the proposed online adaptation outperforms the speaker-independent (SI) model by statistically significant word error rate (WER) or character error rate (CER) reductions of 0.61% and 1.22% absolute (2.99% and 4.48% relative). Real-time factor (RTF) speed-up ratios of up to 9.83 times are obtained over offline batch-mode adaptation.
