Taylor Approximation Variance Reduction for Approximation Errors in PDE-constrained Bayesian Inverse Problems

2025-12-05

AI Summary

In PDE-constrained Bayesian inverse problems, quantifying the Bayesian approximation error (BAE) induced by surrogate models remains computationally prohibitive. To address this, we propose a variance-reduction method based on first-order Taylor expansion: using the linearized PDE solution as a control variate, we offline compute the mean and covariance of the BAE. Crucially, its computational complexity scales with the intrinsic data dimension—not the high-dimensional parameter space—thereby alleviating the Monte Carlo sampling bottleneck in high-dimensional settings. The method requires no additional PDE solves and enables efficient, scalable statistical estimation of the BAE. Experiments on two high-dimensional PDE inverse problems demonstrate that our approach maintains statistical accuracy in BAE quantification while reducing sampling cost by over an order of magnitude compared to standard Monte Carlo estimators.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In numerous applications, surrogate models are used as a replacement for accurate parameter-to-observable mappings when solving large-scale inverse problems governed by partial differential equations (PDEs). The surrogate model may be a computationally cheaper alternative to the accurate parameter-to-observable mappings and/or may ignore additional unknowns or sources of uncertainty. The Bayesian approximation error (BAE) approach provides a means to account for the induced uncertainties and approximation errors (between the accurate parameter-to-observable mapping and the surrogate). The statistics of these errors are in general unknown a priori, and are thus calculated using Monte Carlo sampling. Although the sampling is typically carried out offline the process can still represent a computational bottleneck. In this work, we develop a scalable computational approach for reducing the costs associated with the sampling stage of the BAE approach. Specifically, we consider the Taylor expansion of the accurate and surrogate forward models with respect to the uncertain parameter fields either as a control variate for variance reduction or as a means to efficiently approximate the mean and covariance of the approximation errors. We propose efficient methods for evaluating the expressions for the mean and covariance of the Taylor approximations based on linear(-ized) PDE solves. Furthermore, the proposed approach is independent of the dimension of the uncertain parameter, depending instead on the intrinsic dimension of the data, ensuring scalability to high-dimensional problems. The potential benefits of the proposed approach are demonstrated for two high-dimensional inverse problems governed by PDE examples, namely for the estimation of a distributed Robin boundary coefficient in a linear diffusion problem, and for a coefficient estimation problem governed by a nonlinear diffusion problem.

Problem

• Reduces computational cost of Monte Carlo sampling in Bayesian approximation error approach
• Uses Taylor expansion for variance reduction in PDE-constrained inverse problems
• Ensures scalability for high-dimensional parameter estimation in nonlinear diffusion problems

Innovation

• Taylor expansion reduces Monte Carlo sampling costs
• Linearized PDE solves approximate error statistics efficiently
• Scalable approach independent of uncertain parameter dimension