Regret-Based $(ε,δ)$-optimal Stopping Criteria for Bayesian Optimization

2026-05-21

AI Summary

This work addresses the lack of theoretically grounded stopping criteria in Bayesian optimization, which often leads to excessive function evaluations and no guarantees on solution quality. Focusing on the GP-UCB algorithm, the authors derive a tighter upper bound on instantaneous regret and leverage it to propose the first stopping criterion with $(\varepsilon,\delta)$-optimality guarantees. This criterion ensures that, upon termination, the returned solution is approximately optimal with high probability. Experimental results demonstrate that the proposed method significantly reduces the number of function evaluations while strictly maintaining solution quality, thereby enhancing optimization efficiency.

Abstract

Bayesian optimization (BO) is a widely used iterative black-box optimization method that utilizes Gaussian process (GP) surrogate models. In practice, BO is typically terminated after a fixed evaluation budget is exhausted, which can incur unnecessary cost and provides no optimality guarantee on solution quality. Recent research in developing a practical stopping criterion has made empirical progress, yet a theoretically sound stopping criterion remains a work in progress. In this work, we present provably tighter instantaneous regret bounds for GP upper confidence bound (GP-UCB) at any given iteration. Then, we propose stopping criteria for GP-UCB based on this tighter bound that ensures an $ε$-optimal solution with high probability $1-δ$ upon termination. Numerical experiments are performed to validate and demonstrate the effectiveness and efficiency of our stopping criteria.

Problem

• Bayesian optimization
• stopping criterion
• regret bound
• GP-UCB
• optimality guarantee

Innovation

• Bayesian optimization
• stopping criterion
• regret bound
• GP-UCB
• ε-optimal