Search papers, labs, and topics across Lattice.
This survey paper provides a taxonomy of hallucinations in Video LLMs, categorizing them into dynamic distortion and content fabrication, each with two subtypes. It reviews existing benchmarks, metrics, and mitigation strategies for these hallucinations, highlighting the role of limited temporal representation and visual grounding. The authors then propose future research directions, such as motion-aware encoders and counterfactual learning, to address these limitations.
Video LLMs don't just get details wrong, they fundamentally distort motion and fabricate entire events, demanding a new approach to evaluation and mitigation.
Despite significant progress in video-language modeling, hallucinations remain a persistent challenge in Video Large Language Models (Vid-LLMs), referring to outputs that appear plausible yet contradict the content of the input video. This survey presents a comprehensive analysis of hallucinations in Vid-LLMs and introduces a systematic taxonomy that categorizes them into two core types: dynamic distortion and content fabrication, each comprising two subtypes with representative cases. Building on this taxonomy, we review recent advances in the evaluation and mitigation of hallucinations, covering key benchmarks, metrics, and intervention strategies. We further analyze the root causes of dynamic distortion and content fabrication, which often result from limited capacity for temporal representation and insufficient visual grounding. These insights inform several promising directions for future work, including the development of motion-aware visual encoders and the integration of counterfactual learning techniques. This survey consolidates scattered progress to foster a systematic understanding of hallucinations in Vid-LLMs, laying the groundwork for building robust and reliable video-language systems. An up-to-date curated list of related works is maintained at https://github.com/hukcc/Awesome-Video-Hallucination .
