Search papers, labs, and topics across Lattice.
This paper offers a comprehensive introduction to the application of stochastic differential equations (SDEs) in generative machine learning, emphasizing their probabilistic framework and the Fokker鈥揚lanck equation that describes the evolution of stochastic variables. It derives the evidence lower bound (ELBO) as a foundational concept for discussing various generative modeling techniques, including diffusion models, score matching, and flow matching. A one-dimensional density modeling problem is utilized to illustrate and compare these different parameterizations, highlighting the versatility of the variational approach in generative tasks.
Stochastic differential equations could redefine the landscape of generative modeling by unifying diverse approaches under a single variational framework.
The use of ordinary and stochastic differential equations has led to substantial progress in generative machine learning with applications to, for example, image, video and biomolecule generation. This paper provides a self-contained and informal introduction to the differential equations, the probabilistic framework for using them in generative modeling and the Fokker--Planck equation that governs the temporal evolution of the marginal distribution of the stochastic variables of the differential equations. The variational lower bound on the log-likelihood (the evidence lower bound, ELBO) is derived and used as a general starting point for a discussion of diffusion models, score matching, and flow matching. All of these approaches may be viewed as specific parameterizations of the most general variational approach. A one-dimensional density modeling problem is used as a simple example to compare different parameterizations.
