Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Generative diffusion models exhibit an empirical puzzle where the breakdown of local denoising context closely coincides with the temporal window in which samples commit to high-level semantic classes. Under a mild "common cause" hypothesis where semantic labels account for correlations between distant spatial tokens, the authors theoretically prove that the nonlocality window must be strictly contained within this speciation window. Furthermore, they establish conditions under which both windows collapse into a singular thermodynamic phase transition at scale, verifying this behavior analytically on Gaussian mixture models.
Generative models can only rely on efficient local denoising up to the exact moment they commit to global semantics, after which both phenomena collapse into a single sharp phase transition as system size grows.
The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.