Search papers, labs, and topics across Lattice.
This paper investigates the limitations of vanilla conditional diffusion models in the context of compositional generation, revealing that they often fail to extrapolate to target distributions defined by compositions of source distributions. The authors provide theoretical insights and empirical evidence demonstrating that existing inference-time techniques, including Feynman-Kac correction, do not adequately address the underlying score estimation errors that arise when the target distribution is out-of-distribution. Their findings suggest that a fundamental reevaluation of the approach to compositional generation is necessary to overcome these challenges.
Score estimation errors in vanilla diffusion models can lead to catastrophic failures in compositional generation, revealing a critical gap in current methodologies.
The task of compositional generation involves using a conditional generative model, trained only on a subset of the possible conditions, to produce samples from compositionally-defined target distributions such as a geometric combination of the source distributions. In this work, we argue that this task is often infeasible for vanilla conditional diffusion models: we conjecture that no inference-time technique can efficiently produce samples from the target distribution in certain well-motivated settings. This idea is supported by theory-guided generalization arguments and carefully-designed experiments on both synthetic and realistic data. In particular, while recent methods such as Feynman-Kac correction reduce inference-time approximation error, our results show that score estimation error has a more catastrophic effect on performance when the target distribution is out-of-distribution with respect to the sources, highlighting the need for a different approach to this task.
CMU ML