Search papers, labs, and topics across Lattice.
The paper introduces GenState-AI, a new text-to-video retrieval benchmark using AI-generated videos with controlled state transitions to specifically evaluate temporal reasoning and end-state grounding. The dataset includes hard negatives that differ only in the end-state or semantic content, allowing for fine-grained diagnosis of model confusions. Experiments with MLLM-based baselines reveal a tendency to confuse videos with similar temporal dynamics but incorrect end-states, highlighting limitations in grounding to decisive end-state evidence.
Text-to-video retrieval models struggle to distinguish videos that differ only in their final state, revealing a critical gap in temporal reasoning and end-state grounding.
Existing text-to-video retrieval benchmarks are dominated by real-world footage where much of the semantics can be inferred from a single frame, leaving temporal reasoning and explicit end-state grounding under-evaluated. We introduce GenState-AI, an AI-generated benchmark centered on controlled state transitions, where each query is paired with a main video, a temporal hard negative that differs only in the decisive end-state, and a semantic hard negative with content substitution, enabling fine-grained diagnosis of temporal vs. semantic confusions beyond appearance matching. Using Wan2.2-TI2V-5B, we generate short clips whose meaning depends on precise changes in position, quantity, and object relations, providing controllable evaluation conditions for state-aware retrieval. We evaluate two representative MLLM-based baselines, and observe consistent and interpretable failure patterns: both frequently confuse the main video with the temporal hard negative and over-prefer temporally plausible but end-state-incorrect clips, indicating insufficient grounding to decisive end-state evidence, while being comparatively less sensitive to semantic substitutions. We further introduce triplet-based diagnostic analyses, including relative-order statistics and breakdowns across transition categories, to make temporal vs. semantic failure sources explicit. GenState-AI provides a focused testbed for state-aware, temporally and semantically sensitive text-to-video retrieval, and will be released on huggingface.co.
