Search papers, labs, and topics across Lattice.
This paper introduces SteerVTE, a novel framework for seamless video text editing that leverages a frozen video diffusion model to achieve high precision in text modifications while maintaining stylistic consistency. The method incorporates a lightweight text context adapter with a style encoder and dual-granularity glyph encoders, addressing challenges in cross-frame accuracy and temporal coherence. Extensive experiments reveal that SteerVTE significantly outperforms existing baselines in terms of text accuracy, style consistency, and temporal coherence, making it a breakthrough in video text editing.
SteerVTE achieves unprecedented precision in video text editing, outperforming existing methods in accuracy and style consistency.
Visual text editing aims to precisely modify text in images and videos while preserving stylistic consistency and visual realism. Despite significant advances in the image domain, video text editing remains largely unexplored: it is a localized task demanding stroke-level precision within small text regions, which compounds the challenges of cross-frame accuracy, temporal coherence, and stylistic fidelity. We introduce SteerVTE, a unified framework that \underline{\textbf{steer}}s a frozen video diffusion model to perform precise \underline{\textbf{V}}ideo \underline{\textbf{T}}ext \underline{\textbf{E}}diting through style and glyph control. Built on a frozen diffusion transformer, SteerVTE attaches a lightweight text context adapter with two complementary modules: a style encoder capturing the original text's visual attributes, and dual-granularity glyph encoders encoding the target text at both the line and character levels. To overcome the inherently weak text rendering priors of video foundation models, we further propose a glyph-aware spatial-focal loss and a three-stage progressive training curriculum that scales from image to video data. To support large-scale training, we also develop an automatic synthesis pipeline and construct SteerVTE-1M, a dataset of one million triplets spanning diverse scenes, fonts, and stylistic effects. Extensive experiments demonstrate that SteerVTE substantially outperforms existing video editing baselines across text accuracy, style consistency, and temporal coherence.
