Search papers, labs, and topics across Lattice.
This paper introduces VGIF-Score, an automated framework designed to evaluate spatio-temporal instruction following in video generation models (VGMs). By combining an objective completion branch that analyzes prompts through a Spatio-Temporal Directed Acyclic Graph (ST-DAG) with a subjective satisfaction branch that employs instruction-conditioned AutoRubric, VGIF-Score offers a comprehensive assessment of both completion and perceptual quality. Experiments across 14 VGMs demonstrate that VGIF-Score yields reliable and interpretable evaluations, addressing the limitations of existing evaluation protocols that often lack depth and diagnostic capability.
VGIF-Score reveals that current video generation models struggle with complex instructions, providing a diagnostic lens to pinpoint where they succeed or fail.
Recent video generation models (VGMs) have made substantial progress in visual fidelity, yet their ability to follow long, compositional instructions remains insufficiently evaluated. Existing evaluation protocols often rely on prompts that are short and semantically shallow, with limited atomic constraints and weak spatio-temporal dependencies. They also frequently depend on costly human evaluation or handcrafted vision pipelines, while providing little diagnostic insight into which instruction constraints succeed or fail. To address this gap, we propose VGIF-Score, a highly automated and interpretable framework for evaluating instruction following in video generation. VGIF-Score consists of two complementary components: an objective completion branch that parses prompts into a Spatio-Temporal Directed Acyclic Graph (ST-DAG) and performs dependency-aware QA with short-circuit diagnostics, and a subjective satisfaction branch that uses instruction-conditioned AutoRubric to assess cinematography, visual purity, motion smoothness, and physics adherence. Together, these components produce a unified score that captures both objective completion and perceptual satisfaction. We instantiate this framework on VGIF-Bench, a benchmark of 223 long, structurally entangled prompts paired with approximately 4.3K fine-grained evaluation items. Experiments on 14 proprietary and open-source VGMs across more than 3K generated videos show that VGIF-Score provides reliable, interpretable, and diagnostically useful evaluation of video generation instruction following. The code will be available at https://github.com/PRIS-CV/VGIF-SCORE.
