Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Discrete Gibbs Iterative Neural Sampler (DGInS), a novel fixed-point neural sampler designed for discrete, unnormalized distributions. DGInS overcomes limitations of prior methods by mitigating mode collapse and providing convergence guarantees, unlike previous fixed-point approaches. The framework, built on masked diffusion and extendable to distribution transport, demonstrates scalable, amortized sampling in high-dimensional systems, exemplified by accurate alloy phase diagram estimation.
Discrete neural samplers achieve fixed-point convergence and drastically reduce mode collapse by leveraging Gibbs sampling within a masked diffusion framework.
Sampling from discrete, unnormalized distributions without access to data is a challenging problem. Neural samplers offer a promising approach by training generative models from density evaluations directly. Despite recent progress, existing discrete neural samplers are prone to mode collapse, come without convergence guarantees when trained via fixed-point iterations, and are often tied to a specific reference process such as masked or uniform diffusion. In this work, we introduce Discrete Gibbs Iterative Neural Sampler, a fixed-point neural sampler that addresses these limitations, enabling efficient, scalable learning, substantially reducing mode collapse in practice. Our framework builds on masked diffusion and also extends to transport between pairs of distributions. We demonstrate that the resulting method scales effectively to high-dimensional systems, supports amortized sampling across different conditions, and enables accurate estimation of alloy phase diagrams.
Microsoft Research