Search papers, labs, and topics across Lattice.
This paper introduces a generalized beam search framework for discrete diffusion models that enables parallel candidate generation and modular beam selection objectives. The core contribution is D5P4, a diversity-focused instantiation that formulates beam selection as MAP inference over a Determinantal Point Process, solved with a scalable greedy algorithm. Experiments on text generation and question answering show that D5P4 improves diversity without sacrificing generation quality, offering a controllable trade-off between diversity and model probability.
Discrete diffusion models can now generate more diverse text without sacrificing quality, thanks to a new decoding method that explicitly optimizes for diversity during beam search.
Discrete diffusion models are promising alternatives to autoregressive approaches for text generation, yet their decoding methods remain under-studied. Standard decoding methods for autoregressive models, such as beam search, do not directly apply to iterative denoising, and existing diffusion decoding techniques provide limited control over in-batch diversity. To bridge this gap, we introduce a generalized beam-search framework for discrete diffusion that generates candidates in parallel and supports modular beam-selection objectives. As a diversity-focused instantiation, we propose D5P4, which formulates the selection step as MAP inference over a Determinantal Point Process. Leveraging a scalable greedy solver, D5P4 maintains multi-GPU compatibility and enables an explicit trade-off between model probability and target diversity with near-zero compute overhead. Experiments on free-form generation and question answering demonstrate that D5P4 improves diversity over strong baselines while maintaining competitive generation quality.
