ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping

2026-08-25

AI Summary

本文提出ResiSpec框架，通过调整验证过程中的提议分布来解决多候选方案中因残差漂移导致的效率问题，提高大语言模型服务速度。

Abstract

The efficiency of Large Language Model (LLM) serving is fundamentally limited by the sequential nature of autoregressive decoding. Speculative Decoding (SD) mitigates this by using a lightweight draft model to speculate future tokens, which are then validated by the LLM in a single parallel forward pass. To further boost efficiency, multi-candidate schemes propose diverse candidate sets to increase the likelihood of token acceptance. However, we show that these schemes are bottlenecked by Residual Drift: a phenomenon where the rejection of initial candidates causes the residual target distribution to diverge from the draft model's predictions. This shift renders subsequent candidates ineffective and forces the system into expensive resampling. To resolve this, we propose ResiSpec, a framework that strategically reforms the proposal distribution during verification to anchor the residual target mass within the draft model's high-confidence regions. By mathematically re-aligning the verification process without compromising output exactness, ResiSpec prevents candidate obsolescence and achieves up to 1.92$\times$ speedup over state-of-the-art multi-candidate methods. Code is available at https://github.com/Czzzk/Resispec.

Problem

• Residual Drift
• Speculative Decoding
• Multi-candidate schemes

Innovation

• Residual Drift
• ResiSpec
• proposal distribution
• draft model