Uncertainty-Aware Surrogate-based Amortized Bayesian Inference for Computationally Expensive Models

2025-05-13

AI Summary

For computationally expensive physical or engineering models, traditional Bayesian inference (e.g., MCMC) is infeasible due to prohibitive model evaluation costs, while amortized Bayesian inference (ABI) achieves fast inference but relies on large volumes of high-fidelity simulation data—often impractical to obtain. Although surrogate models alleviate data requirements, their approximation errors frequently induce overconfident posterior estimates. This work proposes the first ABI framework that explicitly models and propagates surrogate uncertainty, integrating Gaussian process or deep surrogates, conditional variational autoencoders (CVAEs), and a principled error propagation mechanism. Evaluated on multiple high-dimensional, expensive models, our method accelerates inference by over two orders of magnitude compared to MCMC, achieves significantly better posterior calibration than standard score-based ABI (SABI) and MCMC, and simultaneously ensures reliability, computational efficiency, and reproducibility.

Technology Category

• Reasoning under Uncertainty: Probabilistic Inference
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Bayesian inference typically relies on a large number of model evaluations to estimate posterior distributions. Established methods like Markov Chain Monte Carlo (MCMC) and Amortized Bayesian Inference (ABI) can become computationally challenging. While ABI enables fast inference after training, generating sufficient training data still requires thousands of model simulations, which is infeasible for expensive models. Surrogate models offer a solution by providing approximate simulations at a lower computational cost, allowing the generation of large data sets for training. However, the introduced approximation errors and uncertainties can lead to overconfident posterior estimates. To address this, we propose Uncertainty-Aware Surrogate-based Amortized Bayesian Inference (UA-SABI) - a framework that combines surrogate modeling and ABI while explicitly quantifying and propagating surrogate uncertainties through the inference pipeline. Our experiments show that this approach enables reliable, fast, and repeated Bayesian inference for computationally expensive models, even under tight time constraints.

Problem

• Bayesian inference requires many costly model evaluations
• Surrogate models introduce approximation errors and uncertainties
• Need for fast, reliable inference for expensive models

Innovation

• Combines surrogate modeling with amortized Bayesian inference
• Quantifies and propagates surrogate uncertainties explicitly
• Enables fast reliable inference for expensive models