Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors analytically characterize the training dynamics of score-based generative models by decomposing the time-integrated score-matching loss across signal-to-noise ratios ($\Lambda(t)$) in high-dimensional multimodal targets. They find that high-SNR regimes learn mode directions uniformly without capturing their relative frequencies, whereas mode weights and amplitude-dependent features are learned exclusively near the "speciation time" where $\Lambda(t) \sim \mathcal{O}(1)$. Consequently, the choice of loss weighting schedule $w(t)$ directly governs the hierarchy and timing of multimodal feature discovery, a mechanism verified empirically on natural images and genomic data.
Diffusion models are completely blind to class frequencies at high signal-to-noise ratios, acquiring relative mode weights exclusively within a narrow $\mathcal{O}(1)$ SNR window known as the speciation time.
Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In practice this drift is modeled by a neural network, trained on a loss integrated over time $t$ with a weighting schedule $w(t)$. Along the backward dynamics, and for multi-modal distributions, trajectories commit to modes of the target within a narrow time window, the \textit{speciation time}. In this work, focusing on high-dimensional data, we decompose the integrated loss into its single-time contributions and analyze each at fixed signal-to-noise ratio $Λ(t)$: we show that $Λ(t)$ sets the rate at which each feature of a multimodal target - the mode directions and their relative weights - is acquired during training. Crucially, at high $Λ(t)$ all mode directions are acquired together, on a single timescale insensitive to their amplitudes, while the relative weights are not learned at all. Only near the speciation time, where $Λ(t)$ becomes of order one, do all features become learnable, each on its own timescale: the weights are acquired jointly with the directions, and the directions at rates set by their relative amplitudes. For models trained on time-integrated objectives, the learning dynamics is then governed by how much of the weighting effectively sits near the speciation time, which provides insights on $w(t)$ design choices. These results follow from an exact high-dimensional analysis of the training dynamics of unbalanced and hierarchical Gaussian mixtures. Numerical experiments on image and human genome haplotype generation recover the predicted hierarchy of learning timescales in more complex settings.