Computational bottlenecks for denoising diffusions

2025-03-11

AI Summary

This paper investigates whether all efficiently sampleable probability distributions can be exactly sampled via diffusion processes governed by polynomial-time-computable drift functions. It identifies efficiently sampleable distributions for which the score-based drift function is computationally intractable—constructing the first counterexamples under the information-computation gap conjecture—and proves that even arbitrarily small perturbations to any polynomial-time-computable drift function cause sampling failure. Method: The analysis integrates statistical learning theory, score-matching optimization, computational complexity characterization, and diffusion process modeling. Contribution/Results: The work establishes a rigorous computational lower bound on diffusion-based sampling feasibility, revealing fundamental computational infeasibility scenarios for score-based generative models. It provides the first complexity-theoretic, formally rigorous characterization of the theoretical limits of such models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Probabilistic Circuits and Graphical Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Content-based information diffusion
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Denoising diffusions provide a general strategy to sample from a probability distribution $mu$ in $mathbb{R}^d$ by constructing a stochastic process $(hat{oldsymbol x}_t:tge 0)$ in ${mathbb R}^d$ such that the distribution of $hat{oldsymbol x}_T$ at large times $T$ approximates $mu$. The drift ${oldsymbol m}:{mathbb R}^d imes{mathbb R} o{mathbb R}^d$ of this diffusion process is learned from data (samples from $mu$) by minimizing the so-called score-matching objective. In order for the generating process to be efficient, it must be possible to evaluate (an approximation of) ${oldsymbol m}({oldsymbol y},t)$ in polynomial time. Is every probability distribution $mu$, for which sampling is tractable, also amenable to sampling via diffusions? We provide evidence to the contrary by constructing a probability distribution $mu$ for which sampling is easy, but the drift of the diffusion process is intractable -- under a popular conjecture on information-computation gaps in statistical estimation. We further show that any polynomial-time computable drift can be modified in a way that changes minimally the score matching objective and yet results in incorrect sampling.

Problem

• Identify computational bottlenecks in denoising diffusion sampling.
• Explore tractability of sampling via diffusions for easy distributions.
• Modify polynomial-time drifts to affect sampling accuracy minimally.

Innovation

• Denoising diffusions sample probability distributions efficiently
• Drift learned via score-matching objective from data
• Polynomial-time drift evaluation ensures efficient sampling