Search papers, labs, and topics across Lattice.
This paper introduces FactorDiff, a novel factor-wise composition framework for discrete diffusion models that enhances compositional generation by dynamically routing smaller factors of samples to the most relevant pre-trained experts. By addressing the limitations of existing methods that treat generated states monolithically, FactorDiff leverages spatial and functional specializations of experts to improve performance on complex reasoning tasks. The approach is validated on the ARC-AGI benchmark, showing that factor-specific routing significantly outperforms traditional global scalar weighting methods in achieving logical consistency and spatial disentanglement.
FactorDiff reveals that dynamically routing sample factors to specialized experts can dramatically enhance reasoning performance in discrete diffusion models.
Discrete diffusion models offer a powerful framework for solving complex reasoning tasks, particularly through compositional generation, which combines multiple pre-trained experts to generalize beyond their individual training data. Recent theoretical corrections introduce time-dependent mixing weights to better align composed diffusion dynamics with the intended target. However, these methods are fundamentally limited by working on a per-sample basis, treating each generated state monolithically and ignoring the potential spatial or functional specializations of different experts. In this work, we address this limitation by proposing FactorDiff - a factor-wise composition framework for diffusion models. We posit that samples can be further decomposed into smaller factors, and propose a sampling process that dynamically routes each factor to the most relevant expert. We instantiate this framework with spatial/pixel-level compositions and validate it on the ARC-AGI benchmark, demonstrating that simple factor-specific routing consistently outperforms complex global scalar weighting schemes on tasks that require logical consistency and spatial disentanglement.
