Search papers, labs, and topics across Lattice.
This study investigates the impact of speech enhancement (SE) on intent classification performance in LLM-based voice systems by introducing a novel metric called Output Divergence Rate (ODR). The findings reveal that while SE can improve perceptual audio quality, it often leads to significant distortions that adversely affect the semantic understanding of downstream tasks, with ODR values indicating substantial intent misclassification. Notably, the MetricGAN{+} approach significantly increases ODR compared to unenhanced noisy speech, highlighting the inadequacy of traditional audio quality metrics in capturing the true performance implications for LLMs.
SE can enhance audio quality but may drastically mislead LLMs, with intent classification errors more than doubling in some cases.
Speech enhancement (SE) is commonly applied as a preprocessing step in spoken AI pipelines under the assumption that better audio quality improves downstream task performance. Whether SE-induced distortions propagate to downstream LLM task performance remains an open question. We introduce Output Divergence Rate (ODR), which measures how often SE changes an LLM's intent classification relative to clean speech, and benchmark five conditions on 2,974 SLURP clips using Whisper large-v3 and wav2vec2-large cascades. Every condition produces ODR significantly above zero ($p < 0.001$, binomial test). MetricGAN{+} more than doubles ODR versus unenhanced noisy speech (0.318 vs. 0.135) despite improving PESQ, and unmitigated echo reaches an ODR of 0.836 through speaker substitution, a failure WER cannot capture. Audio quality metrics range from near-zero to moderate correlation with ODR (SQUIM-MOS $蟻=-0.068$, PESQ $蟻=-0.467$). The MetricGAN{+} and echo results replicate across ASR architectures, indicating that standard audio quality metrics are insufficient for LLM pipeline quality.
