Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Consistent Distribution Matching (CDM), a novel data-free distillation technique for accelerating diffusion and flow models. By unifying sample generation and score estimation within a single student network, CDM eliminates the need for costly teacher rollouts or auxiliary networks. The method achieves state-of-the-art performance, generating high-fidelity images with significantly fewer function evaluations compared to existing data-free distillation approaches.
Data-free diffusion distillation now achieves FID 2.04 at 1-NFE by unifying generation and score estimation in a single student network.
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256$\times$256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.