Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores representation-conditioned diffusion models, using self-supervised representations to improve both unconditional image generation quality and enable controllable image synthesis. They condition a diffusion model on representations extracted from a pre-trained DINO model, finding that this self-conditioning improves FID score. By traversing the latent space of the DINO representations, they demonstrate smooth and disentangled control over generated image attributes.
Unlock controllable image generation without annotated datasets: just steer the latent space of a self-supervised model like DINO.
Diffusion models have emerged as powerful tools for high-quality image generation and editing, but guiding these models to produce specific outputs remains a challenge. Conventional approaches rely on conditioning mechanisms, such as text prompts or semantic maps, which require extensively annotated datasets. In this preliminary work, we explore diffusion models conditioned on representations from a pre-trained self-supervised model. The self-conditioning mechanism not only improves the quality of unconditional image generation, but also provides a representation space that can be used to control the generation. We explore this conditioning space by identifying directions of variations, and demonstrate promising properties in terms of smoothness and disentanglement.