Amortized mean-shift interacting particles

2026-06-14

AI Summary

This work addresses the computational expense of posterior integration in Bayesian inverse problems, where conventional Monte Carlo methods suffer from slow convergence and reliance on numerous forward simulations. The authors propose a learned deterministic integrator that leverages a mapping network to generate signed-weight quadrature nodes from observations and a small set of posterior samples in a single pass, without requiring explicit evaluation of the posterior density or score function. By synergistically combining reweighting—optimal in theory—with node adaptation—empirically beneficial—and integrating amortized inference, conditional normalizing flows, posterior whitening, and dimension-aware kernel functions, the method overcomes scalability limitations in high dimensions. Experiments across diverse posteriors, including a thousand-dimensional groundwater flow problem, demonstrate consistent and significant improvements over independent Monte Carlo samples under any fixed node budget, achieving Pareto dominance.

Abstract

Bayesian inference for inverse problems is run to evaluate integrals -- posterior expectations, tail probabilities, and risks -- across a stream of observations. The standard estimate averages the integrand over posterior samples, a Monte-Carlo average whose error decays only as the square root of the sample size, so accuracy demands many samples -- prohibitive when each one calls a partial-differential-equation forward model. Mean-shift interacting particles need far fewer: they return a small set of signed-weight nodes -- a deterministic quadrature whose weighted averages estimate those integrals. Finding the nodes, however, is a per-observation optimization that, in its most accurate form, reads the posterior score at every step -- returning the cost it meant to save. We introduce amortized mean-shift interacting particles, a learned map that emits the weighted nodes from an observation and a few posterior samples in a single forward pass. Training asks only for joint parameter-observation samples and a posterior to draw from -- a conditional normalizing flow, an empirical conditional, or any reference the user can sample -- and the map learns to integrate that posterior from samples alone, evaluating neither its density nor its score. Once trained, it generalizes to unseen observations and integrands at any node budget and improves on independent samples in two ways: by reweighting them, provably no worse than the equal weights of Monte-Carlo; and by moving them, which empirically lowers it further. Across closed-form, sampled, learned, and physics-based posteriors -- up to a thousand-coefficient groundwater field -- it integrates more accurately than the same number of samples at every budget, and a posterior-whitened, dimension-aware kernel removes the high-dimensional wall. The result is a Pareto improvement on Monte-Carlo integration, not a competitor to drawing more samples.

Problem

• Bayesian inference
• inverse problems
• Monte Carlo integration
• posterior expectations
• computational efficiency

Innovation

• amortized inference
• mean-shift interacting particles
• Bayesian inverse problems
• deterministic quadrature
• posterior integration