Search papers, labs, and topics across Lattice.
This paper introduces DeLS-Spec, a novel decoupled speculative decoding method that enhances the efficiency of block-parallel drafting in large language models (LLMs) by integrating long-context and short-context predictions. Unlike previous methods that require extensive retraining, DeLS-Spec allows for independent training of a lightweight local head, significantly reducing training costs while maintaining flexibility. Experimental results demonstrate that DeLS-Spec outperforms DFlash in terms of speed and average acceptance length across various benchmarks, including math, code, and dialogue tasks.
DeLS-Spec achieves faster inference and longer acceptance lengths by decoupling long and short context predictions, all while slashing training costs.
Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.
