Sampling Decisions

2025-03-17

AI Summary

This work addresses guided sampling from target distributions in discrete space-time settings. Methodologically, it introduces the Decision Flow (DF) framework—a unified generalization of path-integral diffusion and generative flow networks. It is the first to extend continuous-space-time path-integral diffusion to discrete domains; constructs a linearly solvable, neural-network-free DF formulation, where a novel Markov process is defined via convolution of the target distribution with the inverse-time Green’s function; and algorithmically adapts stochastic optimal control principles—specifically Markov decision processes—to enable explicit, closed-form guided sampling. Contributions include: empirical validation on the Ising model demonstrating DF’s efficacy for analytical, neural-network-free sampling; and establishment of a rigorous theoretical foundation for neural-network-augmented guided sampling, thereby bridging stochastic control theory and generative modeling.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

In this manuscript we introduce a novel Decision Flow (DF) framework for sampling from a target distribution while incorporating additional guidance from a prior sampler. DF can be viewed as an AI driven algorithmic reincarnation of the Markov Decision Process (MDP) approach in Stochastic Optimal Control. It extends the continuous space, continuous time path Integral Diffusion sampling technique to discrete time and space, while also generalizing the Generative Flow Network framework. In its most basic form, an explicit, Neural Network (NN) free formulation, DF leverages the linear solvability of the the underlying MDP to adjust the transition probabilities of the prior sampler. The resulting Markov Process is expressed as a convolution of the reverse time Green's function of the prior sampling with the target distribution. We illustrate the DF framework through an example of sampling from the Ising model, discuss potential NN based extensions, and outline how DF can enhance guided sampling across various applications.

Problem

• Develops Decision Flow framework for guided sampling.
• Extends sampling techniques to discrete time and space.
• Enhances sampling accuracy using prior sampler adjustments.

Innovation

• Decision Flow framework for guided sampling
• Extends Diffusion sampling to discrete settings
• Leverages MDP linear solvability for transition adjustments