Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Fleming-Viot Diffusion (FVD), a novel inference-time alignment method for diffusion models that addresses diversity collapse in Sequential Monte Carlo (SMC) samplers. FVD replaces multinomial resampling with a birth-death mechanism inspired by Fleming-Viot population dynamics, incorporating stochastic rebirth noise to handle approximate rewards and prevent deterministic trajectory collapse. Experiments show FVD significantly improves performance on image generation tasks, achieving better ImageReward scores on DrawBench and FID scores on class-conditional tasks, while also being significantly faster than value-based alignment methods.
Fleming-Viot Diffusion (FVD) offers a surprisingly simple and efficient way to align diffusion models at inference time, outperforming existing methods by a large margin while being significantly faster.
We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers. Existing SMC-based diffusion samplers often rely on multinomial resampling or closely related resampling schemes, which can still reduce diversity and lead to lineage collapse under strong selection pressure. Inspired by Fleming-Viot population dynamics, FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment. To handle cases where rewards are only approximately available and naive rebirth would collapse deterministic trajectories, FVD integrates independent reward-based survival decisions with stochastic rebirth noise. This yields flexible population dynamics that preserve broader trajectory support while effectively exploring reward-tilted distributions, all without requiring value function approximation or costly rollouts. FVD is fully parallelizable and scales efficiently with inference compute. Empirically, it achieves substantial gains across settings: on DrawBench it outperforms prior methods by 7% in ImageReward, while on class-conditional tasks it improves FID by roughly 14-20% over strong baselines and is up to 66 times faster than value-based approaches.