Search papers, labs, and topics across Lattice.
This paper introduces MIOH, a fine-grained benchmark designed to evaluate object hallucination in Multimodal Large Language Models (MLLMs) across multi-image scenarios. By systematically assessing four foundational tasks鈥攅xistence, counting, attribute, and position鈥攗nder varying reasoning patterns and adversarial pressures, the study uncovers significant limitations in state-of-the-art models like GPT-5 and Gemini-2.5-Pro, revealing that hallucination issues arise not just from perceptual failures but from challenges in maintaining object representations during integration.
Even leading MLLMs struggle with object hallucination, revealing critical integration-stage weaknesses that standard benchmarks fail to capture.
Multimodal Large Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex reasoning across visual contexts. However, current MLLMs remain fundamentally limited by object hallucination-generating plausible yet factually inconsistent descriptions about objects. Existing benchmarks, designed primarily for single-image settings or providing only high-level multi-image assessments, cannot systematically diagnose how visual complexity and reasoning demands trigger hallucination. To address this gap, we introduce MIOH, a fine-grained multi-image object hallucination benchmark that systematically evaluates object hallucination across four foundational tasks (existence, counting, attribute, position) through three multi-image reasoning patterns (comprehensive, comparative, selective) under three controlled adversarial pressures (visual context scale, perceptual difficulty, contextual bias). Through evaluation of 29 models, we reveal that even state-of-the-art systems like GPT-5 and Gemini-2.5-Pro exhibit distinct failure patterns across different reasoning patterns and tasks. Our evaluation reveals that hallucination stems not merely from perceptual failures but from integration-stage limitations when maintaining object representations across multiple images. MIOH provides a controlled framework for analyzing multi-image object hallucination and serves as a critical evaluation tool for developing more reliable multimodal AI systems.
