Metropolis-Adjusted Diffusion Models

2026-05-10

AI Summary

This work addresses the sampling bias inherent in diffusion models due to time discretization and score function approximation. The authors propose an unbiased Langevin corrector embedded within a predictor–corrector framework, which for the first time integrates a two-coin Bernoulli factory algorithm to enable an exact accept–reject step. They further introduce a Simpson’s rule–based approximation scheme achieving 5/2-order accuracy with negligible additional computational overhead. Notably, the method performs correction using only the score function—without requiring density ratios—and unifies Metropolis–Hastings and Barker acceptance mechanisms. Empirical evaluations on both synthetic and image datasets demonstrate consistent improvements in sample quality, with the proposed approach outperforming baseline methods in terms of FID across image generation tasks.

Abstract

Sampling from score-based diffusion models incurs bias due to both time discretisation and the approximation of the score function. A common strategy for reducing this bias is to apply corrector steps based on the unadjusted Langevin algorithm (ULA) at each noise level within a predictor-corrector framework. However, ULA is itself a biased sampler, as it discretises a continuous diffusion process. In this work, we consider adjusted Langevin correctors that employ Metropolis--Hastings (MH) or Barker's accept-reject steps to correct for this bias. Since the target density ratio typically required by MH-based algorithms is unavailable, we propose methods that instead utilise the score function to compute the correct acceptance probability. We introduce the first exact method for adjusting Langevin corrections in diffusion models, based on a two-coin Bernoulli factory algorithm. We also propose an efficient approximation based on Simpson's rule that achieves accuracy of order $5/2$ in the step size at near-zero marginal cost. We demonstrate that these procedures improve sample quality on both synthetic and image datasets, yielding consistent gains in Fréchet Inception Distance (FID) on the latter.

Problem

• diffusion models
• sampling bias
• Langevin algorithm
• Metropolis-Hastings
• score function

Innovation

• Metropolis-adjusted Langevin
• score-based diffusion models
• bias correction
• Bernoulli factory
• Simpson's rule