Search papers, labs, and topics across Lattice.
This paper addresses the challenge of under-represented modes in diffusion models by introducing a method that combines temperature sampling with variance-corrective time shifting. By adjusting the sampling temperature to flatten dominant modes and lift rare ones, the authors mitigate the variance inflation that typically degrades sample quality. The proposed approach yields consistent improvements in diversity across various pretrained diffusion models, including DiT, Stable Diffusion, and Motion Diffusion, without requiring retraining.
Temperature sampling can enhance mode diversity in diffusion models without sacrificing sample quality, thanks to a novel variance-corrective technique.
Diffusion models faithfully reproduce their training distribution, but also inherit its imbalances and leave rare or under-represented modes hard to reach. A natural inference-time remedy is to sample from the high-temperature target $p^{(纬)}_0(x) \propto p_0(x)^纬$ for $0 < 纬< 1$, which flattens dominant modes and lifts rare ones. However, naive score scaling while correctly reweighting modes also inflates the per-mode variance, breaking the reverse diffusion process and degrading sample quality. We introduce variance-corrective time shifting, a training-free fix that queries the network at a shifted timestep and scales the resulting score by $纬$, canceling the variance inflation while preserving the mode reweighting. The correction turns simple temperature sampling into a practical diversity knob for pretrained diffusion and flow-matching backbones with no retraining, and we demonstrate consistent gains at minimal cost to sample quality and condition fidelity across DiT, Stable Diffusion and Motion Diffusion models. We further show that the timing of the temperature intervention enables coarse-to-fine control: high-noise stages drive compositional diversity across modes, while low-noise stages drive local appearance variation under a fixed composition.
ETH
Google Research