Search papers, labs, and topics across Lattice.
This paper introduces Dependency-Aware Revocable Decoding (DARD), a novel framework for improving the efficiency of diffusion large language model (dLLM) inference by addressing the contamination of verification contexts from unreliable tokens. By categorizing tokens into masked, candidate, and unmasked states, DARD selectively verifies candidate tokens while regulating the influence of less reliable ones, thus enhancing the quality of generated outputs. Experimental results demonstrate that DARD achieves a significant 2.71脳 speedup and a 4.35-point CIDEr score improvement over existing methods, marking a notable advancement in the speed-quality trade-off for dLLMs.
DARD achieves a remarkable 2.71脳 speedup in dLLM inference while enhancing output quality by 4.35 points on CIDEr, redefining the efficiency landscape for large language models.
Diffusion large language models (dLLMs) offer a promising alternative to autoregressive generation by decoding multiple tokens in parallel through iterative denoising. However, increasing decoding parallelism often degrades generation quality, as early errors can contaminate later contexts. Revocable decoding mitigates this issue by re-evaluating decoded tokens and remasking unreliable ones, but existing methods overlook that unreliable tokens may also corrupt the verification context itself. We identify this failure mode and propose Dependency-Aware Revocable Decoding (DARD), a training-free framework that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments across 12 textual and multimodal benchmarks on 3 open-source dLLMs show that DARD consistently improves the speed-quality Pareto frontier over recent revocable decoding methods, achieving a 2.71$\times$ speedup and a 4.35-point CIDEr score gain over Saber on Flickr30K.
