A Convex Approximation Framework for Neural Likelihood-Based Bayesian Inverse Problems

2026-07-07

AI Summary

This work addresses the challenges of high-dimensional Bayesian inverse problems, which often arise due to unknown physical mechanisms, difficulty in quantifying measurement uncertainties, or prohibitive costs of high-fidelity simulations that render conventional inference methods ineffective. The authors propose a neural likelihood approximation framework that directly learns an unnormalized likelihood from data while explicitly incorporating the normalization constant into the training objective. They establish, for the first time, that this learning problem constitutes a strictly convex optimization task. By leveraging KL divergence minimization and empirical risk minimization, they develop a consistency theory guaranteeing uniform convergence of the neural likelihood estimator. Empirical results on image deblurring and nonlinear PDE-based imaging demonstrate that the method stably converges to the true likelihood as sample size increases, significantly enhancing the scalability and robustness of Bayesian inversion.

Abstract

Many problems in science and engineering are difficult to model accurately, either due to unknown physical mechanisms, poorly quantified measurement uncertainty, or prohibitive computational costs of high-fidelity simulations. These challenges limit the applicability of classical probabilistic inference methods such as Markov chain Monte Carlo, especially in high-dimensional Bayesian inverse problems. As data from scientific experiments become increasingly available, machine learning methods offer a flexible alternative to explicit parametric modelling. We study neural likelihood approximation, where the goal is to learn the likelihood function directly from data without explicit knowledge of the underlying data-generating process. A common approach trains likelihood surrogates by minimizing the Kullback-Leibler divergence between the true posterior and an approximate posterior, which is equivalent to minimizing the expected negative log-likelihood. This work improves the theoretical foundations of neural likelihood approximation by alleviating limitations of restrictive model classes: we show that, by working with un-normalized potentials and folding normalization into the training objective, the resulting learning problem is strictly convex. We show that empirical minimizers of the resulting data-driven objective converge to the true likelihood as the sample size grows. Numerical experiments for the neural likelihood approximation are conducted for a deblurring and a non-linear PDE based imaging problem.

Problem

• Bayesian inverse problems
• neural likelihood approximation
• high-dimensional inference
• likelihood-free inference
• scientific machine learning

Innovation

• convex approximation
• neural likelihood
• Bayesian inverse problems
• unnormalized potentials
• posterior consistency