Malliavin-Bismut Score-based Diffusion Models

2025-03-21

AI Summary

This work addresses the fundamental challenge of lacking closed-form expressions for the score function—i.e., the gradient of the log-density—of solutions to stochastic differential equations (SDEs). We propose a novel diffusion generative modeling framework grounded in Malliavin calculus and the Bismut formula. Methodologically, we rigorously establish an analytical connection among Malliavin derivatives, the divergence operator, and the score function. For linear SDEs, we prove that our derived expression coincides exactly with the ground-truth score obtained from the Fokker–Planck equation. Under state-independent diffusion, we further derive, for the first time, a closed-form analytic solution for the score of nonlinear SDEs and uncover a new smoothness structure of the associated probability density. These results provide both theoretical foundations and principled design guidelines for interpretable and analytically tractable diffusion models.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Computer Vision: Diffusion Models for Vision
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

We introduce a new framework that employs Malliavin calculus to derive explicit expressions for the score function -- i.e., the gradient of the log-density -- associated with solutions to stochastic differential equations (SDEs). Our approach integrates classical integration-by-parts techniques with modern tools, such as Bismut's formula and Malliavin calculus, to address linear and nonlinear SDEs. In doing so, we establish a rigorous connection between the Malliavin derivative, its adjoint (the Malliavin divergence or the Skorokhod integral), Bismut's formula, and diffusion generative models, thus providing a systematic method for computing $ abla log p_t(x)$. For the linear case, we present a detailed study proving that our formula is equivalent to the actual score function derived from the solution of the Fokker--Planck equation for linear SDEs. Additionally, we derive a closed-form expression for $ abla log p_t(x)$ for nonlinear SDEs with state-independent diffusion coefficients. These advancements provide fresh theoretical insights into the smoothness and structure of probability densities and practical implications for score-based generative modelling, including the design and analysis of new diffusion models. Moreover, our findings promote the adoption of the robust Malliavin calculus framework in machine learning research. These results directly apply to various pure and applied mathematics fields, such as generative modelling, the study of SDEs driven by fractional Brownian motion, and the Fokker--Planck equations associated with nonlinear SDEs.

Problem

• Derive explicit score functions for SDEs using Malliavin calculus
• Connect Malliavin derivatives with diffusion generative models
• Provide closed-form score expressions for nonlinear SDEs

Innovation

• Uses Malliavin calculus for score function derivation
• Integrates Bismut's formula with classical techniques
• Provides closed-form expressions for nonlinear SDEs