Search papers, labs, and topics across Lattice.
The paper introduces the D2DF framework, which transforms the video object removal process into a one-step generation model by distilling the refinement capabilities of a teacher model into a student model. This approach leverages Prior-Privileged Consistency Distillation (PPCD) and a Self-Guided Fast Planting (SGFP) module to autonomously create scene-consistent pseudo-drafts, eliminating the need for draft dependency. Experimental results demonstrate that the D2DF model achieves state-of-the-art performance in both quality and efficiency, completing the denoising process in approximately one second per video.
Achieving high-fidelity video object removal in just one second, D2DF outperforms traditional methods while eliminating draft artifacts entirely.
Video object removal is a fundamental yet challenging task in video editing. Despite recent progress, existing methods typically fall into two categories. Traditional approaches based on optical flow or attention mechanisms often introduce noticeable artifacts and yield unnatural results. In contrast, diffusion-based methods improve visual realism but demand multiple denoising steps, limiting their practicality. To address these issues, we propose From-Draft-to-Draft-Free (D2DF), a framework that distills the ability of transforming coarse drafts into refined videos into a one-step video generation model. Within D2DF, a teacher model is trained to refine low-quality removal results ("drafts") into high-fidelity videos by multiple steps. Then, through Prior-Privileged Consistency Distillation (PPCD), we distill this capability into a student model that performs one-step removal conditioned on the draft. To eliminate draft dependency, we introduce a Self-Guided Fast Planting (SGFP) module based on our Temporal Masked Transformer that autonomously generates scene-consistent pseudo-drafts in latent space, enabling a fully draft-free one-step model. Extensive experiments show that both draft-conditioned and draft-free versions achieve state-of-the-art performance on multiple metrics, surpassing traditional and multi-step generative methods in both quality and efficiency. The denoising process for a single video takes only about 1 second.
