Search papers, labs, and topics across Lattice.
This paper introduces VidPair-Halluc, a novel benchmark designed to rigorously evaluate video hallucination in large video models (LVMs) by using video pairs with consistent backgrounds but varying foreground semantics. By isolating the effects of hallucination from background variability, the benchmark allows for precise error attribution, which is critical for understanding model limitations. Evaluations reveal that mainstream LVMs struggle with fine-grained video understanding in adversarial contexts, highlighting significant gaps in current capabilities.
Mainstream video models consistently falter in fine-grained understanding, revealing critical vulnerabilities in their hallucination capabilities.
We introduce VidPair-Halluc, a new benchmark for evaluating video hallucination in large video models (LVMs) under rigorous and controlled conditions. Unlike previous benchmarks that primarily rely on text-based perturbations or adversarial questions while neglecting the consistency of visual backgrounds, VidPair-Halluc features video pairs with highly similar backgrounds but distinctly different foreground semantics, enabling precise attribution of model errors to genuine hallucination rather than background variation. The benchmark is constructed through PairFlow, a pipeline that leverages recent advances in text-to-image and video generation to systematically compose stories, generate coherent video clips, and assemble them into adversarial pairs. Covering both spatial and temporal reasoning across ten semantic aspects, VidPair-Halluc comprises 1K high-quality adversarial video pairs and 11K spatio-temporal QA pairs with control over background and foreground variations. Evaluations on mainstream LVMs show persistent difficulty with robust fine-grained video understanding in adversarial settings, and code and data are available at the https://jethrojames.github.io/VidPair-Halluc/.
NUS