DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees

2026-08-13

AI Summary

Existing diffusion-based speculative decoding suffers from limited acceptance rates and acceleration due to its neglect of conditional dependencies among candidate paths. This work proposes a training-free, tree-structured speculative decoding method that extends pretrained autoregressive correction heads from linear chains to tree topologies. By constructing fixed-width candidate trees in batches and employing node scoring with best-first pruning, the approach efficiently selects verification paths. It is the first to apply autoregressive correction mechanisms to tree-structured candidates, decoupling correction inference from sequential heap operations and thereby achieving lossless generation with substantially improved parallel efficiency. Evaluated across seven benchmarks spanning mathematical reasoning, code generation, and dialogue, the method consistently achieves the highest average accepted length—up to 12.97 tokens per step—and acceleration ratios as high as 9.73× across all model–temperature configurations.

Abstract

Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path. Existing recurrent correction incorporates causal information along a single draft chain, whereas diffusion-based tree construction broadens candidate coverage without carrying this correction along individual branches. We introduce DARTree, a training-free speculative decoding method that extends a pretrained AR correction head from chains to trees. DARTree first constructs a fixed-width candidate tree by expanding and scoring all nodes at each depth in a single batch, and then only applies best-first pruning to select the verification tree, decoupling AR-head inference from sequential heap operations. Across seven math, code, and chat benchmarks, DARTree achieves the highest average acceptance length and speedup in all four model--temperature configurations, accepting up to 12.97 tokens per verification round, 98.6\% more than DFlash and 27.9\% more than Domino in the same setting, and reaching up to 9.73$\times$ lossless speedup over locally measured autoregressive decoding.

Problem

• speculative decoding
• diffusion-based drafters
• autoregressive correction
• candidate tree
• token acceptance

Innovation

• speculative decoding
• diffusion-based drafting
• autoregressive correction
• tree-based candidate generation
• training-free acceleration