Search papers, labs, and topics across Lattice.
This paper introduces DBLAST, a dependent block drafting approach that addresses the limitations of existing block and diffusion-style drafters in stochastic speculative decoding. By utilizing a low-rank latent mixture model and an acceptance-oriented training objective, DBLAST effectively improves the accepted draft length in scenarios with higher entropy in the target sampling distribution. Experimental results demonstrate that DBLAST outperforms independent block sampling across various benchmarks, highlighting its robustness in non-greedy decoding contexts.
DBLAST significantly enhances the accepted draft length in stochastic decoding scenarios, particularly when the target distribution's entropy is high.
Speculative decoding accelerates large language models' inference by using a lightweight drafter to propose multiple future tokens and a target model to verify them. While recent block and diffusion-style drafters can predict several positions in a single pass, their training and sampling procedures are typically optimized for greedy decoding or assume that positions in the draft block are conditionally independent. This assumption becomes brittle in non-greedy speculative decoding, where the target distribution is deliberately stochastic and multiple continuations become plausible. We study this mismatch for block diffusion drafters and show that the accepted draft length degrades as the entropy of the target sampling distribution increases. We propose a dependent block drafter based on a low-rank latent mixture over token positions, complemented by an acceptance-oriented training objective that directly targets the expected verified length. Experiments with Qwen3-4B and Qwen3-8B on GSM8K, MT-Bench, HumanEval, and creative-writing benchmarks show that our approach, namely DBLast, consistently improves accepted length over independent block sampling, especially in higher-entropy decoding regimes.
