Search papers, labs, and topics across Lattice.
This paper introduces MotionHalluc, a benchmark designed to diagnose kinematic hallucinations in motion instruction generation for cross-video comparisons. By evaluating state-of-the-art multimodal models on 1540 fine-grained questions across 553 video pairs, the study reveals a significant vulnerability to three types of hallucinations: directional, attributional, and temporal. The authors demonstrate that incorporating explicit quantitative measurements through their Perceive-Parse-Verify (PPV) method leads to an average performance improvement of 10.6%, highlighting the importance of accurate kinematic feedback in motion reasoning tasks.
State-of-the-art multimodal models are highly susceptible to kinematic hallucinations, but a simple measurement injection can boost performance by over 10%.
Motion instruction generation in cross-video comparison aims to produce corrective feedback that describes the differences between a query and a reference motion. However, existing models often generate instructions that exhibit motion hallucinations, failing to reflect actual kinematic differences between paired videos. To systematically investigate these hallucinations, we introduce MotionHalluc, a dedicated benchmark for evaluating motion hallucinations in paired-video comparison. MotionHalluc comprises 1540 fine-grained questions over 553 video pairs, evaluating hallucinations along three core dimensions: (1)directional hallucination, (2)attributional hallucination, and (3)temporal hallucination. Extensive evaluations of state-of-the-art large multimodal models demonstrate high susceptibility to these hallucinations. Furthermore, we provide Perceive-Parse-Verify (PPV) as a training-free measurements extraction and verification baseline that converts candidate instructions into executable measurement queries and supplies kinematic measurements at inference time. Our results show that this simple measurements injection yields an average 10.6% performance gain across models, suggesting that motion reasoning with explicit quantitative measurements is a key factor in reducing hallucinations in cross-video comparison. Our code and dataset will be made publicly available upon acceptance.
