Bayesian Adaptive Calibration and Optimal Design

2024-05-23

arXiv.org

AI Summary

To address the scarcity of calibration data and the tendency of conventional methods to overlook critical information in physics-based modeling, this paper proposes a Bayesian adaptive experimental design framework that jointly optimizes calibration parameter estimation and simulation point selection. The method dynamically selects the most informative simulation points within a batch-sequential process, significantly reducing the number of required simulation evaluations. Its key contributions are: (i) the first application of maximizing the variational lower bound on expected information gain (EIG) for joint inference and design; and (ii) the use of Gaussian processes to jointly model the simulator response, observational noise, and unknown calibration parameters—thereby capturing their intrinsic couplings. Experiments on both synthetic and real-world physical systems demonstrate substantial improvements in calibration accuracy and data efficiency over fixed-design machine learning approaches.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The process of calibrating computer models of natural phenomena is essential for applications in the physical sciences, where plenty of domain knowledge can be embedded into simulations and then calibrated against real observations. Current machine learning approaches, however, mostly rely on rerunning simulations over a fixed set of designs available in the observed data, potentially neglecting informative correlations across the design space and requiring a large amount of simulations. Instead, we consider the calibration process from the perspective of Bayesian adaptive experimental design and propose a data-efficient algorithm to run maximally informative simulations within a batch-sequential process. At each round, the algorithm jointly estimates the parameters of the posterior distribution and optimal designs by maximising a variational lower bound of the expected information gain. The simulator is modelled as a sample from a Gaussian process, which allows us to correlate simulations and observed data with the unknown calibration parameters. We show the benefits of our method when compared to related approaches across synthetic and real-data problems.

Problem

• Computer Simulation
• Model Calibration
• Machine Learning

Innovation

• Bayesian Optimization
• Gaussian Processes
• Calibration Method