Search papers, labs, and topics across Lattice.
HyperDFlash introduces a block-parallel speculative decoding framework specifically designed for the multi-hyper-connection (MHC) architecture of DeepSeek-V4. By optimizing the alignment of residual streams and implementing a lightweight gated residual reducer, the method significantly improves draft accuracy and decoding speed while reducing model parameters. Experimental results demonstrate that HyperDFlash outperforms both the native Multi-Token Prediction module and previous DFlash adaptations across various benchmarks, showcasing its effectiveness in high-performance speculative decoding.
HyperDFlash achieves remarkable decoding speedups and accuracy improvements by aligning residual streams with the MHC architecture, outperforming existing methods.
We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4. Despite the strong initial-token drafting performance of the native Multi-Token Prediction (MTP) module in DeepSeek-V4, its draft accuracy degrades sharply at later positions, as error accumulation from unverified intermediate tokens harms acceptance rates. Although the original DFlash method supports efficient one-pass block drafting, it cannot be seamlessly adapted to the MHC paradigm, since the multi-path residual stream of DeepSeek-V4 induces feature misalignment with conventional drafting designs. To resolve this mismatch, we propose two model-aligned optimizations for MHC residual streams. First, we adopt pre-collapse residual states as the exclusive conditioning signal, preserving multi-path structural information and aligning the drafter with the native prediction pathway of the target model. Second, we replace the heavy generic linear compressor with a lightweight gated residual reducer, whose parameters are inherited from the built-in hyper-connection head. This design yields input-aware path aggregation with three orders of magnitude fewer parameters while maintaining architectural alignment. We further enhance training via a targeted KL distillation loss applied to the LM-head, which regularizes predictions against the full target probability distribution and improves draft quality at early training stages. Experiments across math reasoning, code synthesis, and conversational benchmarks show that HyperDFlash consistently outperforms both the native MTP baseline and vanilla DFlash adaptation. It achieves substantial gains in average accepted draft length and decoding speedup, validating the effectiveness of MHC alignment, gated reduction, and targeted distillation for high-performance speculative decoding.
