Search papers, labs, and topics across Lattice.
This paper introduces a cause-aware error recovery paradigm for cascaded ASR-LLM spoken dialogue systems that disentangles token-level errors into perception, comprehension, and deletion failures using deep ASR latent representations. This fine-grained error diagnosis enables the LLM to orchestrate targeted, multi-turn clarification strategies. Experiments demonstrate that this approach more than doubles recall on domain-shift errors and yields significant improvements in WER and downstream task performance compared to baseline confidence filtering methods.
LLMs can transform ambiguous spoken signals into seamless user interactions by diagnosing the *cause* of ASR errors (perception, comprehension, deletion) and proactively requesting targeted clarification.
Cascaded Automatic Speech Recognition -- Large Language Model (ASR-LLM) pipelines remain popular for industrial Spoken Dialogue Systems (SDS), primarily because their decoupled design ensures perceptual verifiability. However, cascaded systems suffer from error propagation, as transcription failures inevitably cascade to subsequent components, thereby degrading the final interaction quality. Although ASR confidence scores offer a simple filter for unreliable inputs, this approach is fundamentally limited because it typically fails to detect deletion errors or to distinguish between acoustic (inability to hear clearly) and linguistic (inability to understand) mismatches, both of which require targeted recovery strategies. In this paper, we propose a cause-aware error recovery paradigm that fundamentally rethinks robustness in SDS. Unlike traditional confidence filtering, we introduce a suite of small precision-focused detectors that exploit deep ASR latent representations to disentangle token-level errors into perception, comprehension, and deletion failures. This fine-grained diagnostic intelligence empowers the LLM to orchestrate targeted, multi-turn clarification strategies, effectively transforming ambiguous signals into seamless user interactions. Experimental results validate the precision of our approach, which more than doubles the recall on domain-shift errors (57.96% vs. 23.66%) compared to baselines. Crucially, this diagnostic precision yields up to a 30% reduction in WER and a 17% improvement on the downstream task across diverse accents, distortions, and domains.
