🤖 AI Summary
While energy-parameterized diffusion models support Metropolis-Hastings (MH)-based MCMC sampling—yielding substantially improved sample quality under model composition—standard score-parameterized models lack an explicit energy function, preventing direct MH correction. Method: We propose Score-MCMC, a framework that reconstructs energy differences via line-integral approximations of the pre-trained score function, enabling derivation of a computable MH acceptance probability without modifying the underlying score model. Contribution/Results: Score-MCMC theoretically and practically bridges score-based and energy-based MCMC sampling. By coupling the diffusion reverse process with MH correction, it achieves sampling fidelity and diversity on par with energy-parameterized models across multiple benchmarks—particularly enhancing compositional distribution modeling in terms of both accuracy and sample diversity.
📝 Abstract
Diffusion models can be parameterised in terms of either a score or an energy function. An energy parameterisation is appealing since it enables an extended sampling procedure with a Metropolis--Hastings (MH) correction step, based on the change in total energy in the proposed samples. Improved sampling is important for model compositions, where off-the-shelf models are combined with each other, in order to sample from new distributions. For model composition, score-based diffusions have the advantages that they are popular and that many pre-trained models are readily available. However, this parameterisation does not, in general, define an energy, and the MH acceptance probability is therefore unavailable, and generally ill-defined. We propose keeping the score parameterisation and computing an acceptance probability inspired by energy-based models through line integration of the score function. This allows us to reuse existing diffusion models and still combine the reverse process with various Markov-Chain Monte Carlo (MCMC) methods. We evaluate our method using numerical experiments and find that score-parameterised versions of the MCMC samplers can achieve similar improvements to the corresponding energy parameterisation.
