MGD: Moment Guided Diffusion for Maximum Entropy Generation

2026-02-19

AI Summary

This work addresses the slow mixing of conventional MCMC methods in sampling high-dimensional maximum entropy distributions by proposing Moment Guided Diffusion (MGD). Built upon a stochastic interpolation framework, MGD efficiently generates samples from the maximum entropy distribution subject to given moment constraints by solving a stochastic differential equation that guides the sample moments toward the target constraints within finite time. MGD represents the first integration of diffusion generative models with the maximum entropy principle, provably converging to the exact maximum entropy distribution in the large-fluctuation limit. Moreover, it yields a practical estimator for entropy directly computable from dynamical trajectories. Experiments on high-dimensional, multiscale data—including financial time series, turbulent flows, and cosmological fields—demonstrate the method’s effectiveness and scalability through accurate estimation of negative entropy.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Diffusion Models for Vision
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Web Mining and Content Analysis: Content-based information diffusion
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Generating samples from limited information is a fundamental problem across scientific domains. Classical maximum entropy methods provide principled uncertainty quantification from moment constraints but require sampling via MCMC or Langevin dynamics, which typically exhibit exponential slowdown in high dimensions. In contrast, generative models based on diffusion and flow matching efficiently transport noise to data but offer limited theoretical guarantees and can overfit when data is scarce. We introduce Moment Guided Diffusion (MGD), which combines elements of both approaches. Building on the stochastic interpolant framework, MGD samples maximum entropy distributions by solving a stochastic differential equation that guides moments toward prescribed values in finite time, thereby avoiding slow mixing in equilibrium-based methods. We formally obtain, in the large-volatility limit, convergence of MGD to the maximum entropy distribution and derive a tractable estimator of the resulting entropy computed directly from the dynamics. Applications to financial time series, turbulent flows, and cosmological fields using wavelet scattering moments yield estimates of negentropy for high-dimensional multiscale processes.

Problem

• maximum entropy
• moment constraints
• high-dimensional sampling
• generative modeling
• uncertainty quantification

Innovation

• Moment Guided Diffusion
• Maximum Entropy
• Stochastic Interpolant
• Diffusion Models
• Entropy Estimation