Search papers, labs, and topics across Lattice.
SpecFold addresses multi-branch computational redundancy in diffusion language model speculative decoding by introducing token-level residual gating and selective computation reuse within attention and FFN layers. This algorithm-system co-design exploits the high similarity of hidden states across draft branches to significantly reduce verification costs. Empirical results demonstrate up to 1.99x throughput gains over vanilla decoding and 1.64x over Spiffy, without compromising task performance.
Folding multi-branch redundancy slashes speculative decoding costs in diffusion LLMs, unlocking up to 2x faster generation.
Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single forward pass. While prior DLLM acceleration methods primarily exploit temporal redundancy across denoising steps, we identify a complementary redundancy axis within each speculative verification step: multi-branch computational redundancy. During speculative verification, draft branches inherit most tokens from their parents while unmasking a small set of additional positions, causing large portions of hidden states to remain highly similar across branches. We propose SpecFold, an algorithm-system co-design that exploits this multi-branch redundancy to reduce the cost of multi-branch speculative verification. Algorithmically, SpecFold performs token-level residual gating and selectively reuses parent computation through folded attention and FFN while preserving residual hidden states. Systemically, a Triton kernel implementation translates this fine-grained reuse into end-to-end throughput gains through efficient sparse multi-branch execution. SpecFold is orthogonal to temporal caching and compatible with existing DLLM speculation strategies. Across two DLLM families, five models, and five standard benchmarks, SpecFold achieves up to 1.64x throughput over Spiffy and up to 1.99x over vanilla decoding, while maintaining comparable task performance.
