Search papers, labs, and topics across Lattice.
Domino decouples causal modeling from autoregressive drafting in speculative decoding by using a parallel draft backbone to generate preliminary token distributions, which are then refined by a lightweight "Domino head" conditioned on causal prefixes. A base-anchored training curriculum stabilizes teacher-forced causal encoding by initially strengthening the parallel backbone before shifting optimization toward the causally corrected final distribution. Experiments with Qwen3 models demonstrate up to 5.8x throughput speedup under SGLang serving, highlighting the efficiency of decoupling causal modeling.
Achieve up to 5.8x LLM inference speedup by decoupling causal dependency modeling from autoregressive draft execution in speculative decoding, sidestepping the usual trade-off between draft quality and drafting cost.
Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel with the target model. However, its practical speedup is constrained by the trade-off between draft quality and drafting cost: autoregressive drafters model causal dependencies among draft tokens but incur sequential overhead, while parallel drafters reduce drafting cost but weaken intra-block dependency modeling. In this paper, we propose Domino, a speculative decoding framework that decouples causal dependency modeling from expensive autoregressive draft execution. Domino first uses a parallel draft backbone to produce preliminary draft distributions for the entire block, and then applies a lightweight Domino head to refine them with prefix-dependent causal information. To stabilize teacher-forced causal encoding, we further introduce a base-anchored training curriculum that first strengthens the parallel backbone and then gradually shifts optimization toward the causally corrected final distribution. Experiments on Qwen3 models show that Domino achieves up to \(5.49\times\) end-to-end speedup under the Transformers backend and up to \(5.8\times\) throughput speedup under SGLang serving.
