Search papers, labs, and topics across Lattice.
VIS-Ground addresses the challenge of latent dependencies in interactive video storytelling by formulating contextual grounding as transforming heterogeneous inputs into executable constraints. The system employs Structured Context Abstraction, Generation Constraints Induction, and Constrained Video Generation to recover grounded states, project dependencies, and enforce them during generation. Across three backbones, VIS-Ground significantly outperforms baselines, demonstrating improvements in both narrative and knowledge grounding.
Interactive video generation systems can now robustly handle latent dependencies by transforming unstructured inputs into executable constraints, achieving a 10.3-point average improvement.
Video interactive storytelling enables viewers to actively steer how a video unfolds. However, once we allow viewers to intervene during generation, a new challenge arises: The viewer's request can have latent dependencies on both the grounding source and the current rendered video state. These dependencies may not be explicitly stated in any individual input, but emerge only when the source, rendered history, and new viewer intent are considered jointly. Existing interactive video generation systems primarily emphasize following viewer instructions, while source-grounded video generation methods focus on aligning generated content with an external narrative or knowledge source. This leaves a fundamental question underexplored: What context should a generation model ground on during interactive continuation, and how can heterogeneous, unstructured inputs be transformed into such grounding context? In this work, we formulate contextual grounding as the process of transforming heterogeneous input context into an executable constraint model for video generation. To address this challenge, we introduce VIS-Ground, which performs Structured Context Abstraction to recover grounded states and cross-context dependencies, Generation Constraints Induction to project relevant dependencies into candidate-specific constraints, and Constrained Video Generation to enforce these constraints through planning, verification, revision, and rendering. Across three video generation backbones, VIS-Ground consistently achieves the highest overall composite score, reaching an average absolute improvement of 10.3 points over the strongest per-backbone baselines. Detailed analysis further shows gains across both narrative and knowledge grounding, and reveals remaining challenges in dependency extraction, and faithful realization during video rendering.
