Search papers, labs, and topics across Lattice.
This paper introduces Unique Hallucination Pattern (UHP) Detection, a novel framework for identifying hallucinations in large vision-language models (LVLMs) by analyzing structured uncertainty patterns across two axes: perturbation modality and logical polarity. By categorizing hallucinations into four distinct consistency groups, the method extracts both within-group and between-group features to train a lightweight classifier, significantly improving detection performance. Experimental results demonstrate that UHP Detection outperforms existing methods, achieving up to 20.07% AUC-PR gains, and shows strong generalization across different datasets, highlighting the model-specific nature of hallucination behaviors.
Hallucinations in LVLMs reveal a structured uncertainty pattern that can be effectively detected, outperforming traditional methods by up to 20% in precision metrics.
Large vision--language models (LVLMs) demonstrate strong multimodal reasoning capabilities but remain prone to hallucination, where model predictions are not grounded in visual evidence. Existing black-box hallucination detection methods estimate uncertainty through a single consistency metric, implicitly assuming that model uncertainty can be adequately characterized by a single measure. However, hallucinations exhibit diverse manifestations of uncertainty across different behavioral probes, making a single measure insufficient to characterize their underlying behavior. We propose \emph{Unique Hallucination Pattern (UHP) Detection}, a fully black-box framework that models hallucination as a structured uncertainty pattern defined by two axes: perturbation modality (image vs.\ text) and logical polarity (a statement vs.\ its negation). Their intersection produces four complementary consistency groups that capture distinct manifestations of model uncertainty, from which both within-group and between-group features are extracted to train a lightweight classifier. Through comprehensive experiments on AMBER and PhD across three LVLMs, UHP Detection consistently outperforms prior black-box and white-box baselines, with improvements of up to $+18.72\%$ AUC-ROC and $+20.07\%$ AUC-PR over the strongest black-box methods. Extensive ablation studies demonstrate that each consistency group contributes complementary information and that their combination forms a structured hallucination pattern. Furthermore, cross-dataset evaluation shows that this learned pattern generalizes across benchmarks, indicating that hallucination behavior reflects a model-specific consistency pattern. \textbf{Code is publicly available at} https://github.com/amirezzati/uhpdet.
