Diffusion Posterior Sampling is Computationally Intractable

2024-02-20

International Conference on Machine Learning

AI Summary

This work investigates posterior sampling from diffusion models—i.e., drawing samples from ( p(x mid y) ) given an observation ( y ). We establish, under standard cryptographic assumptions (specifically, the existence of one-way functions), that this posterior sampling task is computationally intractable in polynomial time—the first rigorous computational complexity lower bound for diffusion model posterior inference. Methodologically, we employ computational complexity theory and cryptographic reductions to formalize the hardness of posterior sampling as a consequence of one-way function existence, integrating Bayesian inverse problem modeling for precise analysis. Our key contributions are threefold: (1) a rigorous proof of polynomial-time intractability of diffusion posterior sampling; (2) an identification of its fundamental connection to basic cryptographic primitives; and (3) a proof—under the assumption that the forward diffusion process is exponentially invertible—that rejection sampling achieves asymptotic optimality, thereby providing a theoretical benchmark for practical algorithm design.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Applications of cryptography
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Diffusion models are a remarkably effective way of learning and sampling from a distribution $p(x)$. In posterior sampling, one is also given a measurement model $p(y mid x)$ and a measurement $y$, and would like to sample from $p(x mid y)$. Posterior sampling is useful for tasks such as inpainting, super-resolution, and MRI reconstruction, so a number of recent works have given algorithms to heuristically approximate it; but none are known to converge to the correct distribution in polynomial time. In this paper we show that posterior sampling is computationally intractable: under the most basic assumption in cryptography -- that one-way functions exist -- there are instances for which every algorithm takes superpolynomial time, even though unconditional sampling is provably fast. We also show that the exponential-time rejection sampling algorithm is essentially optimal under the stronger plausible assumption that there are one-way functions that take exponential time to invert.

Problem

• Proving computational intractability of diffusion posterior sampling
• Establishing exponential time requirements under cryptographic assumptions
• Showing rejection sampling optimality for inverse problems

Innovation

• Proves computational intractability of posterior sampling
• Links one-way functions to superpolynomial time requirements
• Shows rejection sampling algorithm is essentially optimal