Search papers, labs, and topics across Lattice.
The paper introduces Unified Latents (UL), a framework for learning latent representations jointly regularized by a diffusion prior and decoded by a diffusion model. UL links the encoder's output noise to the prior's minimum noise level, resulting in a training objective that provides a tight upper bound on the latent bitrate. Experiments on ImageNet-512 and Kinetics-600 demonstrate that UL achieves competitive or state-of-the-art generative performance with improved training efficiency compared to existing latent diffusion models.
Ditch Stable Diffusion's latents: Unified Latents (UL) achieves state-of-the-art video generation and competitive image generation with fewer training FLOPs.
We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.
Google Research