Search papers, labs, and topics across Lattice.
This paper introduces HalluScope, a novel framework for fine-grained hallucination diagnosis in Multimodal Large Language Models (MLLMs), addressing the limitations of existing coarse-grained detection methods. By constructing a large-scale dataset, HalluScope-30K, and employing a multi-granular joint reward function, the authors train two models that achieve state-of-the-art performance in hallucination detection and classification. The results demonstrate that fine-grained diagnostic explanations significantly improve the ability of target models to correct hallucinations, showcasing the mutual benefits of detection and classification through joint optimization.
Fine-grained hallucination diagnosis can dramatically enhance the reliability of multimodal language models by revealing the types of hallucinations they produce and how to correct them.
Although Multimodal Large Language Models have achieved strong performance across a wide range of vision-language tasks, they still suffer from hallucinations, where model outputs become inconsistent with the visual content, textual context, or commonsense knowledge. Existing studies primarily address this problem through coarse-grained detection. However, these approaches often provide insufficient diagnostic information for understanding hallucination types and supporting downstream hallucination mitigation. To bridge this gap, we propose fine-grained hallucination diagnosis for MLLMs, a new unified task that jointly performs hallucination detection, classification, and interpretable explanation generation. We develop an automated data generation pipeline and construct HalluScope-30K, a large-scale diagnostic dataset covering eight sources and five task categories. Based on this dataset, we design a multi-granular joint reward function and train two diagnosis models, HalluScope-4B and HalluScope-8B, which achieve state-of-the-art performance on both the MHALO benchmark and our fine-grained hallucination classification benchmark. Notably, detection and classification are mutually beneficial under joint optimization. Furthermore, diagnosis-driven feedback experiments show that the fine-grained diagnostic explanations produced by our model effectively guide target models to correct their hallucinations, with full diagnosis substantially outperforming all baselines on both Qwen3-VL-8B-Instruct and LLaVA-1.5-7B.
