Search papers, labs, and topics across Lattice.
This paper introduces a novel three-stage pipeline for identity-preserving text-to-video generation that addresses the challenges of maintaining a consistent subject identity while executing a sequence of actions. By employing an action-aware prompt polishment stage, an identity-preserving generation stage, and an identity-aware inference enhancement stage, the model effectively mitigates appearance drift across frames. The approach achieved competitive performance, ranking third in the IPVG26 challenge, highlighting its effectiveness in generating coherent and recognizable video sequences from structured action prompts.
Maintaining a consistent identity in sequential-action video generation is possible without sacrificing performance, as demonstrated by a novel three-stage pipeline that ranked third in a competitive challenge.
Identity-preserving text-to-video generation aims to synthesize a video that accurately follows a textual description while maintaining the recognizability of a user-specified subject throughout. The IPVG26 challenge extends this framework from a single holistic prompt to a temporally structured specification. The model additionally receives a sequence of timestamped action captions and must render the subject performing these actions in the specified order. This temporal structure presents a challenge not encountered in previous identity-preserving generation tasks, as the subject must continuously perform a scripted sequence of distinct actions while maintaining a consistent identity. However, end-to-end video generators are prone to appearance drift as motion accumulates and the depicted actions change. We address this challenge with a training-free, three-stage pipeline framework. An action-aware prompt polishment stage first rewrites the inputs into image-generation prompts that specify the terminal state of each action. An identity-preserving generation stage then produces the keyframe sequence by conditioning each frame jointly on the reference identity and its predecessor, thereby decoupling time-invariant appearance from time-varying pose. Finally, an identity-aware inference enhancement stage synthesizes the intermediate segments using multi-reference guidance and identity-driven noise searching, both of which reinforce identity fidelity during sampling. Our method ranked third on the official Track 2 leaderboard, demonstrating competitive performance and strong generality.
