CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credits

2025-10-07

AI Summary

Diffusion large language models (dLLMs) suffer from low initial token confidence during parallel decoding, leading to frequent token re-masking, redundant iterations, and diminished speedup. To address this, we propose the **Trace Credit mechanism**, the first approach to quantify token-level convergence potential as a history-based logits credit score. By accumulating and fusing logits along decoding trajectories in a trajectory-aware manner, Trace Credit accelerates confidence convergence without requiring additional training. Our method significantly improves both the efficiency and stability of parallel decoding: it achieves up to 5.48× throughput acceleration across eight benchmarks, with average generation quality gains of +0.48 (e.g., BLEU, MCQ scores). It scales robustly to long sequences and remains fully compatible with mainstream inference optimizations—including FlashAttention and KV cache compression.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Diffusion large language models (dLLMs) generate text through iterative denoising steps, achieving parallel decoding by denoising only high-confidence positions at each step. However, existing approaches often repetitively remask tokens due to initially low confidence scores, leading to redundant iterations and limiting overall acceleration. Through the analysis of dLLM decoding traces, we observe that the model often determines the final prediction for a token several steps before the decoding step. To leverage this historical information and avoid redundant steps, we introduce the concept of Trace Credit, which quantifies each token's convergence potential by accumulating historical logits. Furthermore, we propose CreditDecoding, a training-free parallel decoding algorithm that accelerates the confidence convergence of correct but underconfident tokens by fusing current logits with Trace Credit. This process significantly reduces redundant iterations and enhances decoding robustness. On eight benchmarks, CreditDecoding achieves a 5.48 times speedup and a 0.48 performance improvement over LLaDA-8B-Instruct, and a 4.11 times speedup with a 0.15 performance improvement over LLaDA-MoE-Instruct. Importantly, CreditDecoding scales effectively to long sequences and is orthogonal to mainstream inference optimizations, making it a readily integrable and versatile solution.

Problem

• Accelerating parallel decoding in diffusion language models
• Reducing redundant iterations during text generation
• Enhancing decoding robustness through historical logit accumulation

Innovation

• Training-free algorithm accelerating diffusion model decoding
• Trace Credit quantifies token convergence using historical logits
• Fuses current predictions with historical data reducing iterations