Accelerating Discrete Diffusion Models with Parallel-In-Time Sampling

2026-07-01

AI Summary

This work addresses the inefficiency of sequential sampling in discrete diffusion models by introducing the first time-parallelized τ-leaping algorithm within a continuous-time Markov chain framework. By leveraging the stochastic integral formulation of τ-leaping and integrating it with Picard iteration, the proposed method enables parallel sampling across time steps. Theoretically, the approach is shown to achieve exponential-order convergence and substantially reduce time complexity. Empirical results demonstrate up to a 9× speedup on synthetic data. In image and text generation tasks, the method maintains comparable generation quality while using only 50% of the function evaluations (NFE), yielding acceleration factors of 1.45–1.86× over baseline approaches.

Abstract

Discrete diffusion models are widely used for learning and generating discrete distributions. As the generation process is inherently sequential, the acceleration of sampling is of significant importance. In this work, we parallelize the mainstream $τ$-leaping algorithm for absorbing discrete diffusion in a Continuous-Time Markov Chain (CTMC) framework. By leveraging the continuous-time stochastic integral form of the $τ$-leaping algorithm and the Picard iteration method, we achieve parallel-in-time sampling acceleration and provide a proof of exponential-factorial convergence for our algorithm. We improve the overall time complexity of $τ$-leaping under absorbing settings from ${\mathcal{O}}(d \log S)$ to ${\mathcal{O}}(\log (d\log S)\cdot \log d)$ with respect to NFE. Empirically, our method shows consistent acceleration across synthetic and real-data settings. The new sampler achieves at most $7$--$9\times$ runtime speedup for synthetic distribution, and maintains the same quality with $50\%$ fewer NFE and $1.45$--$1.86\times$ runtime speedups in image/text tasks on a single GPU. Our research expands the potential of discrete diffusion models for efficient parallel inference, with broader implications for applications such as molecular structure and language generation.

Problem

• discrete diffusion models
• sampling acceleration
• parallel-in-time
• τ-leaping
• Continuous-Time Markov Chain

Innovation

• parallel-in-time sampling
• discrete diffusion models
• τ-leaping
• continuous-time Markov chain
• Picard iteration