Search papers, labs, and topics across Lattice.
EditCtrl introduces a computationally efficient video inpainting framework that decomposes video context processing into local and global components. It achieves efficiency by processing only masked tokens in a local video context module, making computation proportional to the edit size. Guided by a lightweight temporal global context embedder, EditCtrl maintains video-wide consistency while achieving a 10x speedup and improved editing quality compared to full-attention methods.
EditCtrl slashes the compute cost of generative video editing by 10x while *improving* quality, thanks to its focus on local masked tokens and lightweight global context.
High-fidelity generative video editing has seen significant quality improvements by leveraging pre-trained video foundation models. However, their computational cost is a major bottleneck, as they are often designed to inefficiently process the full video context regardless of the inpainting mask's size, even for sparse, localized edits. In this paper, we introduce EditCtrl, an efficient video inpainting control framework that focuses computation only where it is needed. Our approach features a novel local video context module that operates solely on masked tokens, yielding a computational cost proportional to the edit size. This local-first generation is then guided by a lightweight temporal global context embedder that ensures video-wide context consistency with minimal overhead. Not only is EditCtrl 10 times more compute efficient than state-of-the-art generative editing methods, it even improves editing quality compared to methods designed with full-attention. Finally, we showcase how EditCtrl unlocks new capabilities, including multi-region editing with text prompts and autoregressive content propagation.
CMU ML