Score-Based Deterministic Density Sampling

2025-04-25

AI Summary

This work addresses deterministic sampling from an unnormalized target density π, requiring only its log-gradient ∇log π—no pretrained score function or ground-truth samples. We propose the Score-Based Transport Modeling (SBTM) framework, which introduces the first deterministic Wasserstein gradient flow approximation with online learning of time-varying scores. We incorporate relative Fisher information as an intrinsic convergence criterion and support annealing dynamics to handle non-log-concave targets. Theoretically, SBTM matches the exact gradient flow’s relative entropy dissipation rate; empirically, it achieves the optimal O(1/k) convergence rate of Unadjusted Langevin Algorithm (ULA). Generated sample trajectories are smooth, interpretable, and free of stochastic noise. SBTM thus bridges theoretical rigor—guaranteeing asymptotic consistency—with practical flexibility—adapting to complex, non-convex densities without auxiliary models or sampling randomness.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We propose and analyze a deterministic sampling framework using Score-Based Transport Modeling (SBTM) for sampling an unnormalized target density $pi$. While diffusion generative modeling relies on pre-training the score function $ abla log f_t$ using samples from $pi$, SBTM addresses the more general and challenging setting where only $ abla logpi$ is known. SBTM approximates the Wasserstein gradient flow on KL$(f_t|pi)$ by learning the time-varying score $ abla log f_t$ on the fly using score matching. The learned score gives immediate access to relative Fisher information, providing a built-in convergence diagnostic. The deterministic trajectories are smooth, interpretable, and free of Brownian-motion noise, while having the same distribution as ULA. We prove that SBTM dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. We further extend our framework to annealed dynamics, to handle non log-concave targets. Numerical experiments validate our theoretical findings: SBTM converges at the optimal rate, has smooth trajectories, and is easily integrated with annealed dynamics. We compare to the baselines of ULA and annealed ULA.

Problem

• Sampling unnormalized target density without pre-trained score function
• Approximating Wasserstein gradient flow using score matching
• Handling non-log-concave targets via annealed dynamics

Innovation

• Deterministic sampling with Score-Based Transport Modeling
• Learns time-varying score via score matching
• Handles non-log-concave targets via annealing