Search papers, labs, and topics across Lattice.
This paper introduces EquiEdit, a novel video editing framework that effectively balances temporal consistency and editability, two traditionally conflicting objectives in text-guided video editing. The framework employs a temporal Mamba module for enhanced inter-frame consistency and a noise injection strategy utilizing spectral transformation to maintain fidelity while allowing flexible edits. Extensive experiments confirm that EquiEdit significantly improves both consistency and editability compared to existing methods, providing high-quality edited videos that adhere closely to the original input.
Achieving a harmonious balance between temporal consistency and editability in video editing could redefine standards in text-guided video manipulation.
Recently, diffusion models have achieved considerable success in the text-guided video editing domain. However, existing works often struggle to balance the trade-off between temporal consistency and editability in video editing, with consistency and editability typically being inversely related. To address this, we propose a high-quality video editing framework enhanced for consistency and editability, named EquiEdit, which improves coordinatively the temporal consistency and editability of the edited videos while achieving a balance between the two. In terms of temporal consistency, the proposed temporal Mamba module with a tailored temporal-aware scanning scans fused video sequences following four designed directions, effectively enhancing the inter-frame consistency of edited videos. For editability, we design a noise injection strategy based on the spectral transformation to increase editing flexibility, where the Fourier transform is used to preserve the hidden structure in the initial latent noise used for editing, ensuring inter-frame consistency of the edited video and fidelity to the input video. Extensive qualitative and quantitative experiments demonstrate the effectiveness of our method in terms of temporal consistency and editability, as well as its great fidelity to the input video itself.
