Interpolating Discrete Diffusion Models with Controllable Resampling

2026-04-19

AI Summary

Existing discrete diffusion models often suffer from degraded sample quality in text and graph generation due to early irreversible errors or excessive reliance on intermediate latent states. To address this, this work proposes IDDM, a novel framework incorporating a controllable resampling mechanism that reallocates part of the probability mass to the marginal distribution during generation, thereby reducing dependency on intermediate states and enabling more effective token correction. IDDM employs an interpolation-based Markov transition mechanism that dynamically balances between preserving the current state, resampling from the prior, and flipping toward the target, while strictly maintaining marginal consistency. Notably, the approach fully decouples training from inference. Experimental results demonstrate that IDDM achieves state-of-the-art performance among discrete diffusion models on both molecular graph and text generation tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Discrete diffusion models form a powerful class of generative models across diverse domains, including text and graphs. However, existing approaches face fundamental limitations. Masked diffusion models suffer from irreversible errors due to early unmasking, while uniform diffusion models, despite enabling self-correction, often yield low-quality samples due to their strong reliance on intermediate latent states. We introduce IDDM, an Interpolating Discrete Diffusion Model, that improves diffusion by reducing dependence on intermediate latent states. Central to IDDM is a controllable resampling mechanism that partially resets probability mass to the marginal distribution, mitigating error accumulation and enabling more effective token corrections. IDDM specifies a generative process whose transitions interpolate between staying at the current state, resampling from a prior, and flipping toward the target state, while enforcing marginal consistency and fully decoupling training from inference. We benchmark our model against state-of-the-art discrete diffusion models across molecular graph generation as well as text generation tasks, demonstrating competitive performance.

Problem

• discrete diffusion models
• masked diffusion
• uniform diffusion
• intermediate latent states
• error accumulation

Innovation

• discrete diffusion models
• controllable resampling
• marginal consistency
• error accumulation mitigation
• interpolating generative process