Search papers, labs, and topics across Lattice.
This paper introduces \textsc{LESS}, a training-free adaptive sampling method for diffusion language models (dLLMs) that optimizes token commitment by treating it as an online stopping problem. By implementing mutual-stability sampling, \textsc{LESS} ensures that a masked position is only unmasked when it meets specific stability criteria, significantly enhancing the efficiency of the decoding process. Evaluated on various models and benchmarks, \textsc{LESS} achieves an average accuracy improvement while reducing reverse steps by 72.1%, leading to lower inference latency and computational costs.
Achieving 72.1% fewer reverse steps without sacrificing accuracy, \textsc{LESS} redefines efficiency in diffusion language models.
Diffusion large language models (dLLMs) offer a promising alternative to autoregressive decoding by iteratively refining masked sequences, enabling parallel token updates and bidirectional conditioning. Their practical efficiency, however, is limited by sampling procedures that execute a fixed number of reverse denoising steps selected before decoding, spending computation on already-stable positions and sometimes committing unstable ones too early. We present \textsc{LESS}, a training-free, model-agnostic adaptive sampler that treats token commitment as an online stopping problem. \textsc{LESS} implements mutual-stability sampling through a joint stability rule that makes a masked position eligible for unmasking only when its top-1 prediction has high confidence, its top-1 token persists across recent reverse steps, and its predictive distribution is stable under top-$K$ inter-step Jensen--Shannon divergence. We evaluate \textsc{LESS} on Dream-7B, LLaDA-8B, and LLaDA-1.5-8B, covering full-sequence diffusion and semi-autoregressive blockwise sampling regimes, across seven benchmarks spanning general knowledge, math, and code. \textsc{LESS} improves average accuracy over strong training-free adaptive samplers while using $72.1\%$ fewer reverse steps than fixed-budget decoding. Since each reverse step requires a Transformer forward pass, these step-count reductions translate into fewer forward evaluations, lower measured wall-clock latency, and lower estimated inference compute.
