Search papers, labs, and topics across Lattice.
This paper introduces a novel approach called "cached LLM probability retrieval," which enhances automatic speech recognition (ASR) by leveraging local teacher LLMs to provide next-token probabilities for context-target pairs. The method operates without the need for retraining existing acoustic models, allowing for seamless integration into current ASR systems while significantly improving performance. Evaluations demonstrate that this approach outperforms traditional 1-pass ASR in 28 out of 39 scenarios, achieving lower non-oracle error rates, particularly at a context length of 8.
Cached LLM probability retrieval can improve ASR performance without the heavy lifting of retraining, outperforming traditional methods in a majority of tested scenarios.
Large language models (LLMs) enhance automatic speech recognition (ASR) by providing linguistic priors; however, their direct rescoring is costly because it requires evaluating every N-best hypothesis. This paper introduces"cached LLM probability retrieval,"which involves querying a local teacher LLM offline to obtain next-token probabilities for ASR-relevant context-target pairs. These probabilities are then utilized during recognition via cache lookups, backoff strategies, and optional scoring for significant misses. The method is training-free and can integrate with existing recognizers without requiring modifications to acoustic models. Evaluations across various ASR models reveal that cached retrieval outperforms 1-pass ASR in 28 of 39 settings and achieves lower non-oracle errors. Context length analysis indicates that benefits peak at a context length of 8, suggesting that cached probability retrieval is an effective and lightweight ASR adaptation method, in contrast to the heavy training required for Generative Error Correction (GER) or knowledge distillation (KD).
