Discrete Diffusion with Sample-Efficient Estimators for Conditionals

2026-02-23

AI Summary

This work proposes a novel denoising diffusion framework for generative modeling in discrete state spaces that circumvents the need to approximate discrete score functions. The approach directly drives the reverse diffusion process using pointwise conditional probabilities and integrates a round-robin noising mechanism with the highly sample-efficient Neural Interaction Screening Estimator (NeurISE) for conditional modeling. Experimental evaluations on the Ising model, binarized MNIST, and data generated by quantum annealers demonstrate that the method substantially outperforms existing discrete diffusion models across multiple metrics—including total variation distance, cross-correlation, and kernel density estimation—yielding more accurate and efficient generation of discrete data.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Probabilistic Circuits and Graphical Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We study a discrete denoising diffusion framework that integrates a sample-efficient estimator of single-site conditionals with round-robin noising and denoising dynamics for generative modeling over discrete state spaces. Rather than approximating a discrete analog of a score function, our formulation treats single-site conditional probabilities as the fundamental objects that parameterize the reverse diffusion process. We employ a sample-efficient method known as Neural Interaction Screening Estimator (NeurISE) to estimate these conditionals in the diffusion dynamics. Controlled experiments on synthetic Ising models, MNIST, and scientific data sets produced by a D-Wave quantum annealer, synthetic Potts model and one-dimensional quantum systems demonstrate the proposed approach. On the binary data sets, these experiments demonstrate that the proposed approach outperforms popular existing methods including ratio-based approaches, achieving improved performance in total variation, cross-correlations, and kernel density estimation metrics.

Problem

• discrete diffusion
• generative modeling
• conditional estimation
• sample efficiency
• discrete state spaces

Innovation

• Discrete Diffusion
• Sample-Efficient Estimation
• Single-Site Conditionals
• NeurISE
• Generative Modeling