BayeSQP: Bayesian Optimization through Sequential Quadratic Programming

2026-02-03

AI Summary

This work proposes a novel framework that integrates sequential quadratic programming (SQP) with Bayesian optimization to address high-dimensional black-box constrained optimization problems. The method employs a second-order Gaussian process surrogate model to jointly capture the objective and constraint functions, along with their gradients and Hessian matrices, explicitly quantifying the associated uncertainties. Search directions are generated by solving uncertainty-aware second-order cone subproblems, followed by a constraint-aware Thompson sampling strategy for line search. By incorporating the SQP structure into Bayesian optimization—leveraging higher-order information in a principled manner—this approach achieves superior performance over state-of-the-art methods on multiple high-dimensional benchmark tasks. The study establishes a flexible and scalable paradigm for synergistically combining classical optimization techniques with modern Bayesian approaches.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Constraint Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We introduce BayeSQP, a novel algorithm for general black-box optimization that merges the structure of sequential quadratic programming with concepts from Bayesian optimization. BayeSQP employs second-order Gaussian process surrogates for both the objective and constraints to jointly model the function values, gradients, and Hessian from only zero-order information. At each iteration, a local subproblem is constructed using the GP posterior estimates and solved to obtain a search direction. Crucially, the formulation of the subproblem explicitly incorporates uncertainty in both the function and derivative estimates, resulting in a tractable second-order cone program for high probability improvements under model uncertainty. A subsequent one-dimensional line search via constrained Thompson sampling selects the next evaluation point. Empirical results show thatBayeSQP outperforms state-of-the-art methods in specific high-dimensional settings. Our algorithm offers a principled and flexible framework that bridges classical optimization techniques with modern approaches to black-box optimization.

Problem

• black-box optimization
• Bayesian optimization
• sequential quadratic programming
• Gaussian process
• high-dimensional optimization

Innovation

• Bayesian optimization
• Sequential Quadratic Programming
• Gaussian process surrogates
• Second-order cone programming
• Black-box optimization