Search papers, labs, and topics across Lattice.
This paper investigates the relationship between forward marginal error and numerical stability in diffusion sampling, revealing that small forward-marginal errors do not ensure stability in the reverse-time process. The authors construct a smooth score field that maintains low forward-marginal $L^2$ error while demonstrating divergent moments in its Euler--Maruyama discretizations. Their findings highlight that weak convergence can occur despite diverging Wasserstein distances, challenging existing assumptions about the reliability of score matching in diffusion models.
Small forward-marginal errors can mask significant numerical instability in diffusion sampling, leading to misleading conclusions about model performance.
Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. We show that small forward-marginal error does not guarantee numerical stability. We construct a single smooth score field with arbitrarily small forward-marginal $L^2$ error. The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler--Maruyama discretizations converge in probability while every positive moment diverges. Thus weak convergence can hold even though every Wasserstein distance $W_p$, $p\ge1$, diverges. The same failure can occur within one fixed finite neural architecture. We construct a family of bounded, globally Lipschitz denoisers for which both the forward-marginal error and the path-space total variation distance tend to zero, while their Euler--Maruyama endpoints diverge in every $W_p$. For compactly supported data, we also give a simple positive result. Projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, gives grid-uniform moment bounds, and yields Wasserstein convergence under mild local regularity. Experiments with a small fixed DiT-style network show large growth along rare numerical trajectories and its suppression by denoiser projection, while overall trajectory errors remain small.
