Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Masked diffusion language models (dLLMs) struggle with generation diversity due to confidence-based samplers deferring unmasking of high-entropy fork positions. This paper introduces Fork-dLLM, a hybrid sampler that selectively employs autoregressive (AR) ordering only at uncertain fallback steps, preserving parallel generation otherwise. Fork-dLLM achieves AR-level pass@k scaling with 2-3x greater efficiency, and its extension to reinforcement learning, ForkGRPO, matches or exceeds AR-based GRPO performance at reduced training cost.
Parallel diffusion language model generation can match autoregressive diversity and efficiency by strategically falling back to sequential sampling only at high-entropy decision points.
Masked diffusion language models (dLLMs) have shown strong potential for faster inference through parallel token generation when combined with confidence-based samplers. However, recent work has shown that such methods can defer unmasking high-entropy fork positions at which multiple plausible continuations exist. This results in reduced generation diversity, as shown by worse pass@k scaling, and limits gains obtainable from RL post-training. To avoid this flexibility trap, prior work advocated for autoregressive (AR) sampling. Here, we show that discarding confidence-based sampling is unnecessary and, once inference cost is taken into account, wasteful. We first propose Fork-dLLM, a simple hybrid sampler that uses AR-style ordering only at uncertain fallback steps while retaining parallel generation otherwise. We then extend the same principle to post-training with ForkGRPO, which uses Fork-dLLM rollouts and applies the GRPO objective only at fallback steps, preserving exact policy-likelihood ratios while substantially reducing rollout and optimization cost. In our experiments, Fork-dLLM matches the strong pass@k scaling of AR sampling while being 2-3x more efficient, and ForkGRPO achieves downstream performance comparable to or better than AR-based GRPO baselines at a substantially lower training cost.