Tweedie's Formulae and Diffusion Generative Models Beyond Gaussian

2026-05-19

AI Summary

This work addresses the limitation of existing diffusion models, which predominantly rely on Gaussian noise and struggle to capture non-Gaussian data distributions with state-dependent diffusion coefficients. The paper presents the first systematic generalization of Tweedie’s formula to non-Gaussian diffusion processes—including geometric Brownian motion, squared Bessel processes, and Cox-Ingersoll-Ross processes—and derives corresponding denoising score matching objectives. By extending the theoretical foundation of diffusion models beyond the Gaussian paradigm, the proposed framework demonstrates strong empirical performance across diverse applications such as image generation, synthetic financial time series modeling, and empirical Bayes parameter estimation, highlighting its versatility and practical potential.

Abstract

Diffusion models have achieved remarkable success in generating samples from unknown data distributions. Most popular stochastic differential equation-based diffusion models perturb the target distribution by adding Gaussian noise, transforming it into a simple prior, and then use denoising score matching, a consequence of Tweedie's formula, to learn the score function and generate clean samples from noise. However, non-Gaussian diffusion models with state-dependent diffusion coefficient have been largely underexplored, as have the corresponding Tweedie's formulae. In this work, we extend Tweedie's formula to important non-Gaussian processes, including geometric Brownian motion (GBM), squared Bessel (BESQ) processes, and Cox-Ingersoll-Ross (CIR) processes, thereby yielding the corresponding denoising score-matching objectives. We then apply the derived formulae to image and financial time series generation using GBM- and CIR-based diffusion models, and to empirical Bayes estimation under the BESQ setting. The reported experimental results demonstrate the potential of non-Gaussian models.

Problem

• Tweedie's formula
• non-Gaussian diffusion
• diffusion generative models
• state-dependent diffusion
• score matching

Innovation

• Tweedie's formula
• non-Gaussian diffusion
• score matching
• geometric Brownian motion
• Cox-Ingersoll-Ross process