Bandits attack function optimization

2026-05-05

AI Summary

This work addresses black-box function optimization under a limited budget of function evaluations by formulating it as a constrained sequential decision-making problem and proposes the Simultaneous Optimistic Optimization (SOO) algorithm. SOO integrates a deterministic partitioning strategy with a multi-armed bandit framework, leveraging an optimistic exploration mechanism to effectively balance global exploration and local exploitation—all without requiring gradient information. Experimental results on the CEC’2014 single-objective real-parameter benchmark suite demonstrate that SOO achieves competitive performance in both solution quality and numerical efficiency. The approach thus establishes a theoretically grounded and practically viable paradigm for gradient-free black-box optimization.

Abstract

We consider function optimization as a sequential decision making problem under budget constraint. This constraint limits the number of objective function evaluations allowed during the optimization. We consider an algorithm inspired by a continuous version of a multi-armed bandit problem which attacks this optimization problem by solving the tradeoff between exploration (initial quasi-uniform search of the domain) and exploitation (local optimization around the potentially global maxima). We introduce the so-called Simultaneous Optimistic Optimization (SOO), a deterministic algorithm that works by domain partitioning. The benefit of such approach are the guarantees on the returned solution and the numerical efficiency of the algorithm. We present this machine learning approach to optimization, and provide the empirical assessment of SOO on the CEC'2014 competition on single objective real-parameter numerical optimization test-suite.

Problem

• function optimization
• budget constraint
• exploration-exploitation tradeoff
• multi-armed bandit
• sequential decision making

Innovation

• Simultaneous Optimistic Optimization
• multi-armed bandit
• function optimization
• domain partitioning
• exploration-exploitation tradeoff