Decoupled Contrastive Decoding via Expert-Aligned Drafting

2026-08-13

AI Summary

This work addresses the high inference overhead of existing contrastive decoding (CD), which relies on amateur models, and the uncertainty in speculative decoding regarding alignment between draft models and contrastive signals. The authors propose Decoupled Contrastive Decoding (DCD), which shifts the contrastive signal entirely to the verification stage and employs a lightweight, feature-level drafter aligned with the expert model to generate candidate tokens. This approach preserves the original CD output distribution while significantly improving efficiency. By integrating the EAGLE3 framework, Cross-alpha training for alignment, and an approximate dual-drafter decomposition, DCD achieves 1.65–1.95× speedup over vanilla CD greedy decoding on 8B models and reduces proposal-path latency by 5–12× on MMLU, demonstrating for the first time that expert-aligned drafting outperforms contrastive-aware drafting.

Abstract

Contrastive Decoding (CD) improves generation quality, but its amateur-model pass makes decoding expensive. Accelerating CD with speculative decoding raises a proposal-alignment question: should the contrastive signal shape the drafter, or should it remain only in verification? We study this question in the lightweight feature-level drafter regime. Two controlled diagnostics, matched Cross-alpha training and an Approximate Dual-Drafter decomposition, give the same diagnosis: contrastive-aware drafting does not consistently improve over expert-aligned drafting because the contrastive correction is usually weaker than drafter error, and reconstruction can amplify that error. We introduce Decoupled Contrastive Decoding (DCD), which drafts with an expert-aligned lightweight proposer and applies the amateur only in unchanged CD verification. Standard speculative verification preserves the vanilla-CD output distribution. Across the main 8B settings, EAGLE3-based DCD achieves average greedy speedups of 1.65 to 1.95x over vanilla CD and reduces MMLU proposal-path latency by about 5 to 12x relative to amateur-coupled proposal paths.

Problem

• Contrastive Decoding
• Speculative Decoding
• Drafter Alignment
• Generation Efficiency
• Amateur-Expert Modeling

Innovation

• Decoupled Contrastive Decoding
• speculative decoding
• expert-aligned drafting
• contrastive decoding
• lightweight drafter