Search papers, labs, and topics across Lattice.
This paper addresses the issue of exposure bias in diffusion models by introducing a method called Spectral Alignment (SPA), which calibrates the power spectrum of predictions to a pre-computed prior. The authors identify frequency-dependent discrepancies between training and inference that lead to error accumulation, demonstrating that fixed correction rules are ineffective due to varying mismatches across models and timesteps. SPA, which incurs minimal computational overhead and enhances performance across various architectures, shows consistent improvements in reducing exposure bias during iterative sampling.
Frequency-dependent discrepancies in diffusion models reveal that fixed correction rules fail, but a lightweight spectral alignment method can effectively reduce exposure bias.
Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency-dependent discrepancies between training and inference, which can be interpreted as frequency-dependent SNR error. Crucially, the direction of this mismatch varies across models and timesteps, indicating that fixed correction rules do not generalize. We propose Spectral Alignment (SPA), a lightweight, guidance-based method that calibrates the power spectrum of intermediate predictions to a pre-computed prior. Our approach consists of two stages: (1) offline fitting of a parametric spectrum model from training data, and (2) inference-time guidance via efficient FFT-based gradient computation. SPA introduces minimal computational overhead (3-4\%) and is complementary to Classifier-Free Guidance (CFG). We demonstrate consistent improvements across diverse architectures, from pixel-space models (DDPM, ADM) to latent diffusion models (SD2.0, SDXL) and flow-matching models (SD3.5, FLUX). Our implementation is available at https://github.com/SonyResearch/SPA.
