Search papers, labs, and topics across Lattice.
This paper explores the integration of diffusion pruning and step distillation to optimize the inference cost of diffusion models. By introducing a teacher-aligned repair stage, the authors successfully bridge the gap between pruning and distillation, allowing for effective one-step distillation from a pruned generator. The results show that a 20% pruned model can achieve an FID of 3.12 with significantly reduced parameters and evaluations, outperforming the original model's efficiency while maintaining high image quality.
A novel teacher-aligned repair stage enables significant parameter reduction in diffusion models while preserving image quality, achieving an FID of 3.12 with just one network evaluation.
Diffusion models generate high-quality images, but their inference cost comes from two sources: large denoising networks and repeated denoising steps. Existing compression pipelines usually attack these costs separately. Pruning reduces the network, but most pruning methods still rely on a long post-pruning retraining stage to recover a many-step sampler. Step distillation reduces the number of denoising steps, but it usually assumes a student that can already follow the teacher well enough to receive useful distillation gradients. This paper asks whether post-pruning retraining can be replaced by step distillation. We find that the direct replacement fails: after pruning an EDM2-XS teacher, starting SiDA from the pruned checkpoint produces unusable samples. We introduce a short teacher-alignment repair stage as a bridge between pruning and step distillation. The bridge matches the pruned generator to the teacher on noisy real-image latents, then hands the repaired checkpoint to one-step distillation. On ImageNet-512, the original EDM2-XS baseline uses 124.713M parameters and 63 network evaluations, reaching an FID of 3.53. With a suitable distillation objective, our 20% pruned one-step generator uses 98.826M parameters and one network evaluation, reaching an FID of 3.12. With 30% pruning, the model uses 88.029M parameters and one network evaluation, with an FID of 4.26.
