Search papers, labs, and topics across Lattice.
This tutorial elucidates diffusion models by framing them through the lens of differential equations, starting with conditional Gaussian forward processes and deriving both ODE and SDE representations. It demonstrates how these forward processes transport the data distribution to a Gaussian prior, and subsequently derives the corresponding reverse-time dynamics governed by the marginal score. The tutorial then connects the standard noise-prediction objective with score matching and discusses sampling methods, including DPM-Solver, classifier guidance, and classifier-free guidance, while also relating DDPM and DDIM to the reverse SDE/ODE framework.
Diffusion models, often seen as mysterious, are elegantly demystified by viewing them through the prism of differential equations, revealing their underlying structure and unifying seemingly disparate approaches like DDPM and DDIM.
This tutorial develops diffusion models from the viewpoint of differential equations. We begin with the conditional Gaussian forward process and show that this path admits both an ordinary differential equation (ODE) representation and a stochastic differential equation (SDE) representation. Averaging the conditional process over the data distribution then yields marginalized forward ODE and SDE formulations that transport the data distribution $p_0=p_{\mathrm{data}}$ to a Gaussian prior $p_1=\mathcal{N}(0,I)$. We next derive the corresponding reverse-time dynamics, namely the reverse SDE and the reverse probability-flow ODE, both of which are governed by the marginal score $\grad\log p_t(x)$. This leads to a training objective for score estimation and shows that the standard noise-prediction objective is equivalent to score matching up to an additive constant independent of the model parameters. We then discuss sampling methods for the learned reverse dynamics, including DPM-Solver, as well as guided sampling through classifier guidance and classifier-free guidance. Finally, we compare DDPM and DDIM with the reverse SDE/ODE framework and show that they share the same training objective, while DDPM sampling corresponds to discrete reverse-SDE sampling and DDIM sampling corresponds to reverse-ODE sampling.
