HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction

2026-06-25

AI Summary

This work addresses the challenge of native multi-token prediction (MTP) in DeepSeek-V4’s Multi-HyperConnection (MHC) architecture, where error accumulation across intermediate tokens severely degrades draft accuracy and conventional speculative decoding fails to align with its residual stream. To overcome this, we propose a block-parallel speculative decoding framework tailored for MHC, which leverages pre-collapsed residual states as conditioning signals and introduces a lightweight gated residual reducer to enable efficient draft generation aligned with the target model’s structure. We further enhance draft quality and training stability through a novel target-aware KL distillation loss and a parameter inheritance strategy. Evaluated on mathematical reasoning, code generation, and dialogue tasks, our method substantially outperforms native MTP and DFlash, achieving significantly higher average accepted lengths and decoding speedups.

Abstract

We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4. Despite the strong initial-token drafting performance of the native Multi-Token Prediction (MTP) module in DeepSeek-V4, its draft accuracy degrades sharply at later positions, as error accumulation from unverified intermediate tokens harms acceptance rates. Although the original DFlash method supports efficient one-pass block drafting, it cannot be seamlessly adapted to the MHC paradigm, since the multi-path residual stream of DeepSeek-V4 induces feature misalignment with conventional drafting designs. To resolve this mismatch, we propose two model-aligned optimizations for MHC residual streams. First, we adopt pre-collapse residual states as the exclusive conditioning signal, preserving multi-path structural information and aligning the drafter with the native prediction pathway of the target model. Second, we replace the heavy generic linear compressor with a lightweight gated residual reducer, whose parameters are inherited from the built-in hyper-connection head. This design yields input-aware path aggregation with three orders of magnitude fewer parameters while maintaining architectural alignment. We further enhance training via a targeted KL distillation loss applied to the LM-head, which regularizes predictions against the full target probability distribution and improves draft quality at early training stages. Experiments across math reasoning, code synthesis, and conversational benchmarks show that HyperDFlash consistently outperforms both the native MTP baseline and vanilla DFlash adaptation. It achieves substantial gains in average accepted draft length and decoding speedup, validating the effectiveness of MHC alignment, gated reduction, and targeted distillation for high-performance speculative decoding.

Problem

• speculative decoding
• multi-hyper-connection
• draft accuracy
• feature misalignment
• block-parallel decoding

Innovation

• speculative decoding
• multi-hyper-connection
• gated residual reduction
• MHC alignment
• KL distillation