Search papers, labs, and topics across Lattice.
This paper introduces Cross-Space Distillation, a method that enables knowledge transfer from high-capacity diffusion models (Teachers) to compact, deployment-friendly models (Students) despite differences in latent resolution and VAE parameterization. The authors present the Bridge, a lightweight latent interface that facilitates this transfer by mapping Student latents into Teacher space without altering the Student's architecture. Experimental results demonstrate that this approach significantly enhances the performance of compact models, exemplified by an improvement in SD 1.5 from 5.4 to 9.4 HPSv3 while maintaining efficient inference and compatibility with existing ecosystems.
Compact diffusion models can now leverage the power of high-capacity Teachers without architectural changes, achieving remarkable performance gains.
Modern one-step diffusion models achieve impressive quality through distribution-based timestep distillation. Yet, they rely on a critical assumption: Teacher and Student must inhabit the same latent space. This Shared-Space constraint prevents knowledge transfer from modern high-capacity Teachers (e.g., SD 3.5 and Flux) into compact, deployment-friendly Students such as SD 1.5, whose latent resolution and VAE parameterization differ from the Teacher. We formalize this overlooked regime as Cross-Space Distillation, where Teacher and Student differ in both latent resolution and VAE space. To enable distillation under this mismatch, we introduce the Bridge, a lightweight latent interface that maps Student latents into the Teacher space without modifying the Student backbone. Bridge combines a frozen Student VAE decoder as a spatial prior with a compact learnable projector, and is trained with latent reconstruction and attention fidelity objectives for stable Teacher-space alignment. Across diverse modern Teachers, Bridge enables substantial gains for compact one-step Students; for example, it improves SD 1.5 from 5.4 to 9.4 HPSv3 while preserving one-step inference, low latency, and broad ecosystem compatibility. These results show that heterogeneous large Teachers can be distilled into efficient, deployable backbones through a lightweight latent-space interface.
