Search papers, labs, and topics across Lattice.
This paper introduces a structured suffix modeling method for accelerating Diffusion Language Models (DLMs) by segmenting the suffix into local, middle, and tail regions, each retaining a different number of tokens based on their structural significance. By incorporating information from previous decoding steps into the current suffix representations, the method enhances the efficiency of DLM inference without requiring additional training. Empirical evaluations reveal that this approach can achieve up to a 72.81脳 speedup in long-sequence inference when integrated with existing acceleration techniques, significantly improving performance across multiple benchmarks.
Achieving up to a 72.81脳 speedup in long-sequence inference could redefine efficiency standards for Diffusion Language Models.
Diffusion Language Models (DLMs) exhibit strong parallel decoding capabilities by denoising multiple tokens in a single generation step. However, this parallelism comes with substantial computational overhead, as each step requires interactions with all suffix tokens. Existing methods typically reduce this cost by retaining only a local suffix window as a substitute for the full suffix. Despite their effectiveness, these methods overlook the structural heterogeneity across suffix regions and re-initialize suffix tokens with identical representations at each timestep. To this end, we propose a structured suffix modeling method for efficient DLM inference. Specifically, we divide the suffix into three regions, i.e., the local, middle, and tail regions, and retain different numbers of suffix tokens in each region according to their structural roles. Moreover, we incorporate the decoding results from the previous step into the suffix token representations at the current step, allowing them to carry evolving denoising information across generation steps. Notably, our method is training-free and orthogonal to several existing acceleration techniques, such as parallel decoding strategies and KV cache. Empirical results across multiple benchmarks on three DLMs demonstrate that our method can further accelerate DLM inference and improve performance in most cases. In particular, in long-sequence inference, our method achieves up to a \(72.81\times\) speedup when combined with other acceleration techniques. Our code is available at https://github.com/zifengcheng/SSM.
NUS