Bayesian Optimization of Partially Known Systems using Hybrid Models

2026-03-11

AI Summary

This work addresses the slow convergence and low sample efficiency of standard Bayesian optimization in high-dimensional nonlinear systems by proposing a hybrid Bayesian optimization framework that integrates physics-based models with Gaussian processes. The method embeds known physical equations into the Gaussian process prior to construct a physics-informed surrogate model, which is then combined with sample average approximation to solve constrained nonlinear stochastic programming problems. Evaluated on a single-stage distillation simulation optimization task, the proposed approach achieves convergence within a single iteration—substantially outperforming standard Bayesian optimization, which fails to converge within 25 iterations—thereby demonstrating markedly improved sample efficiency and superior design performance.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Bayesian optimization (BO) has gained attention as an efficient algorithm for black-box optimization of expensive-to-evaluate systems, where the BO algorithm iteratively queries the system and suggests new trials based on a probabilistic model fitted to previous samples. Still, the standard BO loop may require a prohibitively large number of experiments to converge to the optimum, especially for high-dimensional and nonlinear systems. We present a hybrid model-based BO formulation that combines the iterative Bayesian learning of BO with partially known mechanistic physical models. Instead of learning a direct mapping from inputs to the objective, we write all known equations for a physics-based model and infer expressions for variables missing equations using a probabilistic model, in our case, a Gaussian process (GP). The final formulation then includes the GP as a constraint in the hybrid model, thereby allowing other physics-based nonlinear and implicit model constraints. This hybrid model formulation yields a constrained, nonlinear stochastic program, which we discretize using the sample-average approximation. In an in-silico optimization of a single-stage distillation, the hybrid BO model based on mass conservation laws yields significantly better designs than a standard BO loop. Furthermore, the hybrid model converges in as few as one iteration, depending on the initial samples, whereas, the standard BO does not converge within 25 for any of the seeds. Overall, the proposed hybrid BO scheme presents a promising optimization method for partially known systems, leveraging the strengths of both mechanistic modeling and data-driven optimization.

Problem

• Bayesian optimization
• partially known systems
• hybrid models
• expensive-to-evaluate systems
• physics-based constraints

Innovation

• Bayesian optimization
• hybrid modeling
• Gaussian process
• physics-informed constraints
• sample-average approximation