Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models

2024-07-11

arXiv.org

AI Summary

To address fundamental challenges in Bayesian posterior inference—including poor uncertainty calibration, parameter non-identifiability, and intractable high-dimensional integration—this paper introduces *Inflation Flow*: an invertible, deterministic ODE-based flow grounded in diffusion modeling, which maps observed data to a low-dimensional Gaussian latent space while preserving local geometry and exact invertibility. We reformulate the diffusion process as an interpretable, differentiable ODE, coupled with a tailored noise schedule and rigorously invertible numerical integration, thereby ensuring theoretically bounded approximation error and correct propagation of uncertainty. Experiments demonstrate that Inflation Flow yields well-calibrated, dimensionally compact, and expressive posterior estimates across multiple benchmarks. Notably, it is the first method to simultaneously resolve the overconfidence and non-identifiability pitfalls inherent in variational inference, establishing a new principled paradigm for Bayesian inference.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Inference
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Beyond estimating parameters of interest from data, one of the key goals of statistical inference is to properly quantify uncertainty in these estimates. In Bayesian inference, this uncertainty is provided by the posterior distribution, the computation of which typically involves an intractable high-dimensional integral. Among available approximation methods, sampling-based approaches come with strong theoretical guarantees but scale poorly to large problems, while variational approaches scale well but offer few theoretical guarantees. In particular, variational methods are known to produce overconfident estimates of posterior uncertainty and are typically non-identifiable, with many latent variable configurations generating equivalent predictions. Here, we address these challenges by showing how diffusion-based models (DBMs), which have recently produced state-of-the-art performance in generative modeling tasks, can be repurposed for performing calibrated, identifiable Bayesian inference. By exploiting a previously established connection between the stochastic and probability flow ordinary differential equations (pfODEs) underlying DBMs, we derive a class of models, inflationary flows, that uniquely and deterministically map high-dimensional data to a lower-dimensional Gaussian distribution via ODE integration. This map is both invertible and neighborhood-preserving, with controllable numerical error, with the result that uncertainties in the data are correctly propagated to the latent space. We demonstrate how such maps can be learned via standard DBM training using a novel noise schedule and are effective at both preserving and reducing intrinsic data dimensionality. The result is a class of highly expressive generative models, uniquely defined on a low-dimensional latent space, that afford principled Bayesian inference.

Problem

• Big Data
• Bayesian Inference
• Uncertainty Estimation

Innovation

• Diffusion Models
• Dimensionality Reduction
• Bayesian Inference