Search papers, labs, and topics across Lattice.
This paper introduces Parallel Decoding Distillation (PDD), a novel trajectory-based distillation method designed to accelerate video and image generation in diffusion models by enabling multiple denoising steps per network evaluation. By avoiding the complexities of variational score distillation and adversarial losses, PDD not only simplifies the training process but also significantly enhances video diversity while maintaining state-of-the-art performance with fewer function evaluations. The method demonstrates superior results on various benchmarks, achieving high-quality outputs with 4-8 function evaluations, thus addressing the computational inefficiencies of existing approaches.
Achieving state-of-the-art video generation with significantly improved diversity and speed, PDD redefines the efficiency of diffusion models.
Generation in video diffusion or flow models is computationally expensive due to the slow and iterative sampling process. Current state-of-the-art (SOTA) acceleration methods heavily rely on variational score distillation (VSD) and adversarial losses to distill diffusion models into few-step generators. Albeit achieving high-quality video generation, these training losses are notoriously hard to optimize and suffer from mode collapse, leading to loss of video diversity and lack of motion. In this paper, we introduce Parallel Decoding Distillation (PDD), a simplified and scalable trajectory-based distillation method for fast inference of diffusion and flow matching models. Our architecture and training procedure are compatible with any pre-trained model and support sampling with a varying number of function evaluations (NFE). PDD accelerates generation by predicting multiple denoising steps per network evaluation. Conceptually, it learns a representation of the mean velocity without regressing its derivative using JVPs or finite-difference approximations. Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. Moreover, PDD presents a significant improvement in generated video diversity.
NVIDIA