🤖 AI Summary
This work addresses the limitations of traditional Bayesian inference, which relies on exact likelihoods and suffers when the likelihood is misspecified, intractable, or misaligned with the target discrepancy. The authors propose the first integration of Sinkhorn divergence as a generalized Bayesian loss within Hamiltonian Monte Carlo (HMC) and its adaptive variant NUTS, accommodating both balanced and unbalanced optimal transport settings. They further incorporate common random numbers to handle stochastic simulators and introduce a heuristic for hyperparameter selection that preserves gradient consistency with Hamiltonian dynamics. Empirical evaluations on Gaussian models, noisy spiral manifolds, pulse misalignment, and CIFAR-10 image patch alignment demonstrate the method’s effectiveness, robustness, and the nuanced differences between transport mechanisms.
📝 Abstract
Bayesian posterior sampling is a ubiquitous paradigm for problems where a point estimate of parameters is not sufficient, such as risk analysis and uncertainty quantification. However, likelihoods may be misspecified, intractable, computationally expensive, or not representative of the discrepancy of interest. Generalized Bayes extends likelihood-based posterior updates by using other losses. Sinkhorn divergences have appealing geometric properties: they compare empirical measures directly and yield smooth gradients thanks to entropic regularization. In this work, we introduce Sinkhorn divergences as Generalized Bayes losses for Hamiltonian Monte Carlo (HMC) and No-U-Turn Sampler (NUTS). We also propose heuristics to set hyperparameters that affect the stability and calibration quality, such as the number of Sinkhorn iterations, the entropic regularization strength, and the marginal relaxation penalty. In regimes where the forward model relies on a stochastic simulator, we combine HMC/NUTS with a common-random-numbers strategy to obtain a deterministic surrogate objective that preserves gradients and Hamiltonian dynamics. We study both mass-preserving balanced and relaxed unbalanced settings. We evaluate our method empirically on (1) a simple Gaussian model as a sanity check; (2) a distribution supported on a noisy spiral manifold where a likelihood-based approach is a poor fit; (3) a Gaussian pulse model with misalignment due to errors-in-variables, emphasizing robustness to misspecification; and (4) CIFAR-10 image patch alignment under perturbations, highlighting differences between balanced and unbalanced regimes.