Search papers, labs, and topics across Lattice.
This paper introduces Vera, a layered diffusion model designed for content-preserving video editing that addresses the challenge of unwanted alterations in static elements during video regeneration. By generating an edit layer and an alpha matte for compositing, Vera effectively separates the creative editing process from the preservation of original content. Evaluated against leading models, Vera demonstrates superior content preservation while maintaining competitive edit quality, leveraging a high-quality dataset of 486K frames with accurate alpha mattes.
Vera achieves superior content preservation in video editing by generating separate edit layers, ensuring that static elements remain unchanged while still allowing for creative modifications.
Video diffusion models have enabled remarkable progress in video generation and editing. However, content preservation remains a core challenge: existing methods regenerate every pixel and often alter elements that should remain unchanged, such as characters or background scenes. We introduce Vera, a layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design. To encourage coherent composition with the source video, we extend the text-to-video DiT into a Mixture-of-Transformers (MoT) architecture, with separate DiTs for each layer that interact through joint self-attention. To support the training of Vera, we further construct a high-quality layered dataset with accurate alpha mattes, diverse scenes and dynamics, and visual effects. Across our quantitative benchmark and human preference study, Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.
