Search papers, labs, and topics across Lattice.
This paper introduces MoHallBench, a benchmark specifically designed to evaluate motion hallucination in Video Large Language Models (VideoLLMs), addressing a critical gap in existing benchmarks that primarily focus on object hallucination and coarse action perception. The benchmark includes 11,306 video clips and 40,493 question-answer pairs, systematically assessing three sources of hallucination: co-occurrence priors, sequential inference, and similarity confusion. Experimental results reveal a significant disconnect between action recognition performance and resistance to hallucination, with sequential inference being the most problematic source, suggesting that current models often over-infer actions from incomplete motion cues.
VideoLLMs may excel at recognizing actions but often hallucinate human motions, revealing a critical flaw in their design.
Video Large Language Models (VideoLLMs) have shown strong progress in video understanding, yet they still suffer from hallucinations that are inconsistent with visual evidence. Existing benchmarks mainly focus on object hallucination or coarse action perception, leaving a key video-specific problem underexplored: motion hallucination, in which models infer human motions that are absent from the video. We present MoHallBench, a benchmark for diagnosing motion hallucination in VideoLLMs. MoHallBench systematically evaluates three major sources of hallucination: co-occurrence priors, sequential inference, and similarity confusion. It contains 11,306 video clips and 40,493 question-answer pairs, covering binary-choice, multiple-choice, and generative settings. We further introduce a bi-directional questioning protocol with bias-aware metrics to reduce affirmation bias in binary evaluation. Experiments on ten recent open-source VideoLLMs reveal a clear decoupling between action recognition and hallucination resistance, as models that perform well on positive action recognition often fail on adversarial negatives. Among all settings, sequential inference hallucination is the most severe, showing that current models tend to over-infer expected outcomes from partial motion cues. Our analyses further confirm that stronger priors and finer-grained similarity substantially amplify hallucination. We hope MoHallBench can facilitate future evaluation and mitigation of motion hallucination in VideoLLMs.
