Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks

2026-06-25

AI Summary

Current evaluation protocols for generative systems exhibit significant biases when assessing hybrid approaches that combine autoregressive and diffusion-based decoding. This work proposes Speculative Refinement (SpecRef), a training-free hybrid decoding method that employs entropy-guided selective masking to generate draft outputs with an autoregressive model, subsequently refined by a masked diffusion language model. Through multi-faceted evaluation—encompassing execution pass rate, exact match, and log-likelihood—we uncover critical blind spots in existing benchmarks across six datasets: code benchmarks conflate structural discovery with logical correctness, multi-stage refinement degrades performance, log-likelihood poorly correlates with generation quality, and Python post-processing disproportionately disrupts non-autoregressive models. Notably, incorporating syntactic scaffolding boosts code accuracy from near zero to over 20%, underscoring the profound impact of evaluation protocols on reported results.

Abstract

How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding? We study this question through Speculative Refinement (SpecRef), a training-free hybrid method that warm-starts a masked diffusion language model from an AR draft using entropy-guided selective masking. Evaluating SpecRef across six benchmarks (HumanEval, MBPP, GSM8K, BBH, ARC-Challenge, HellaSwag) with three distinct evaluation protocols (execution-based pass@1, exact-match, log-likelihood scoring), we surface several findings relevant beyond our specific system: (1) code benchmarks conflate structural discovery with logical correctness: providing a syntactic scaffold lifts accuracy from near zero to over 20% without changing the model, indicating that much of the baseline failure is structural; (2) a refinement tension phenomenon where multi-stage correction degrades already-correct tokens, exposing benchmark saturation ceilings invisible to single-model evaluation; (3) log-likelihood and generative evaluation produce different model rankings for the same model pair, suggesting they measure different capabilities; (4) standard Python post-processing silently breaks code evaluation for non-AR generators. These observations apply to any multi-stage or non-autoregressive generation pipeline and point toward more diagnostic evaluation practices.

Problem

• autoregressive decoding
• diffusion decoding
• evaluation protocols
• generation systems
• benchmarking

Innovation

• Speculative Refinement
• hybrid decoding
• diffusion language model
• entropy-guided masking
• multi-stage generation