Search papers, labs, and topics across Lattice.
DyLLM accelerates masked diffusion language model (MDLM) inference by exploiting temporal sparsity in token representations across denoising steps. It identifies salient tokens based on the cosine similarity of attention contexts between adjacent steps, recomputing feed-forward and attention operations only for these tokens. This training-free framework achieves up to 9.6x throughput improvement on reasoning and code-generation tasks while maintaining the accuracy of models like LLaDA and Dream.
MDLMs can be sped up by nearly 10x without retraining, simply by focusing computation on the tokens that actually change between denoising steps.
Masked Diffusion Language Models (MDLMs) enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative denoising process remains computationally expensive because it repeatedly processes the entire sequence at every step. We observe that across these diffusion steps, most token representations remain stable; only a small subset, which we term salient tokens, contributes meaningfully to the next update. Leveraging this temporal sparsity, we present DyLLM, a training-free inference framework that accelerates decoding by selectively computing only these salient tokens. DyLLM identifies saliency by measuring the cosine similarity of attention contexts between adjacent denoising steps. It recomputes feed-forward and attention operations only for salient tokens while reusing cached activations for the remainder. Across diverse reasoning and code-generation benchmarks, DyLLM achieves up to 9.6x higher throughput while largely preserving the baseline accuracy of state-of-the-art models like LLaDA and Dream.
