Search papers, labs, and topics across Lattice.
This paper introduces ASRD (Anchor Supervised Revocable Decoding), a novel framework designed to enhance the performance of Diffusion Large Language Models (dLLMs) by addressing the issues of error propagation and local error reinforcement during the decoding process. By utilizing trusted Anchor Tokens identified through temporal consistency, ASRD effectively separates reliable context from uncertain candidates, leading to improved generation quality. Experimental results show that ASRD not only surpasses existing remasking strategies by achieving accuracy gains of up to 6.4% but also accelerates inference throughput by as much as 7.2 times.
ASRD leverages trusted Anchor Tokens to dramatically improve decoding quality and speed in diffusion LLMs, achieving a 6.4% accuracy boost while accelerating inference by 7.2x.
Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: \textit{Error Propagation}, where new tokens absorb toxic information from erroneous context, and \textit{Local Error Reinforcement}, where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted \textit{Anchor Tokens}, which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4\% while accelerating inference throughput by up to 7.2$\times$.
