Search papers, labs, and topics across Lattice.
This paper introduces a method for detecting hallucinations in SpeechLLMs at inference time by analyzing attention maps, avoiding the need for costly gold-standard outputs. They derive four attention-based metrics (AUDIORATIO, AUDIOCONSISTENCY, AUDIOENTROPY, and TEXTENTROPY) to capture pathological attention patterns linked to hallucinations. Experiments on Qwen-2-Audio and Voxtral-3B show their approach outperforms existing baselines, achieving up to +0.23 PR-AUC improvement in-domain and generalizing to out-of-domain ASR settings, particularly when using a subset of attention heads.
SpeechLLMs' hallucinations betray themselves in their attention patterns, offering a new way to detect these errors without needing expensive human-labeled data.
Hallucinations in Speech Large Language Models (SpeechLLMs) pose significant risks, yet existing detection methods typically rely on gold-standard outputs that are costly or impractical to obtain. Moreover, hallucination detection methods developed for text-based LLMs do not directly capture audio-specific signals. We investigate four attention-derived metrics: AUDIORATIO, AUDIOCONSISTENCY, AUDIOENTROPY, and TEXTENTROPY, designed to capture pathological attention patterns associated with hallucination, and train lightweight logistic regression classifiers on these features for efficient inference-time detection. Across automatic speech recognition and speech-to-text translation tasks, evaluations on Qwen-2-Audio and Voxtral-3B show that our approach outperforms uncertainty-based and prior attention-based baselines on in-domain data, achieving improvements of up to +0.23 PR-AUC, and generalises to out-of-domain ASR settings. We further find that strong performance can be achieved with approximately 100 attention heads, improving out-of-domain generalisation compared to using all heads. While effectiveness is model-dependent and task-specific training is required, our results demonstrate that attention patterns provide a valuable tool for hallucination detection in SpeechLLMs.
Amazon Science