Search papers, labs, and topics across Lattice.
This paper introduces amortized moment matching, a novel approach that leverages neural networks to learn data moments as distributional training signals, enhancing the efficiency of diffusion denoisers through polynomial projections. The method establishes the Amortized Fr茅chet Distance (AMFD) loss, which dynamically learns conditional moments without the need for explicit marginal moment calculations, allowing for scalable application to high-dimensional data. Empirical results demonstrate that AMFD significantly improves training dynamics and one-step generation performance on ImageNet, while also enhancing instruction-following capabilities in text-to-image generation tasks compared to existing models.
AMFD not only outperforms traditional moment matching methods but also enables unprecedented gains in instruction-following for text-to-image generation.
We propose amortized moment matching, utilizing neural networks to learn data moments as distributional training signals. By casting diffusion denoisers through polynomial projections, we establish a general framework for moment amortization, revealing that an $n$-th degree projection explicitly identifies data moments up to order $n+1$. Derived from the tractable affine case, we instantiate the Amortized Fr茅chet Distance (AMFD) loss. Unlike FD-loss which relies on explicit marginal moment calculations, AMFD is able to dynamically learn conditional moments via an alternating, matrix-free optimization pipeline that effortlessly scales to high-dimensional data. When operating on global representation features, AMFD serves as a powerful post-training objective; empirically, its neural formulation yields more robust training dynamics than exact statistical matching, substantially surpassing the FD baseline on the FDr$^6$ metric and achieving superior one-step generation on ImageNet. Furthermore, it unlocks direct exploration within native generative spaces, suggesting that the first two moments can identify target distributions only in spaces with strong semantics. Finally, when scaled to text-to-image generation, the condition-aware nature of AMFD unlocks massive gains in instruction-following capabilities, enabling our one-step models to outperform their multi-step FLUX.2 [klein] 4B teachers on the GenEval benchmark while achieving on-par performance on PickScore. Code and checkpoints are available at https://github.com/poppuppy/amfd.
