Search papers, labs, and topics across Lattice.
This study conducts a comprehensive empirical analysis of pixel-space diffusion models, revealing that direct large-scale pre-training in pixel space converges significantly slower than in latent space. To address this, the authors propose a latent-to-pixel strategy that efficiently acquires generative priors in latent space before transitioning to pixel space during post-training. Their findings culminate in a practical recipe that enables pixel-space models to match or exceed the performance of latent-space models while achieving substantial end-to-end inference speedups of 3.18 to 4.75 times.
Pixel-space diffusion models can now rival latent-space counterparts while achieving up to 4.75 times faster inference.
This paper investigates an increasingly important topic in generative modeling: pixel-space diffusion models. Although numerous studies have explored this topic, most focus on small-scale or class-conditional settings. Consequently, a practical recipe for training pixel-space models that rival or exceed well-established latent-space counterparts remains elusive. Through a comprehensive empirical study, we first observe that direct large-scale pre-training in pixel space converges substantially more slowly than in latent space. This observation motivates a latent-to-pixel strategy that acquires generative priors efficiently in latent space and transitions to pixel space during post-training. We then systematically investigate the key design choices governing this transition, including weight initialization, data composition, prediction target, decoder architecture, and noise schedule, and identify a practical recipe that makes the resulting pixel-space models match or outperform their latent-space counterparts while delivering 3.18 to 4.75 times end-to-end inference speedups. We hope that our findings provide useful empirical insights and practical guidelines for future research on pixel-space generation.
