🤖 AI Summary
Existing batch Bayesian optimization (BO) methods suffer significant performance degradation as batch size increases, failing to fully exploit parallel computing resources. To address this scalability bottleneck, we propose a novel paradigm for large-scale parallel BO: it decomposes the high-dimensional search space into orthogonal, axis-aligned low-dimensional subspaces and introduces the first-of-its-kind Expected Subspace Improvement (ESI) acquisition function, which jointly optimizes diverse yet convergent batch query points within each subspace. This approach effectively balances exploration and exploitation while enabling scalable parallelization. Empirical evaluation on standard benchmarks demonstrates that our method substantially outperforms sequential BO in wall-clock time and consistently achieves state-of-the-art or competitive performance among seven leading batch BO algorithms. The implementation is publicly available in MATLAB, confirming both efficiency and practical applicability.
📝 Abstract
Extending Bayesian optimization to batch evaluation can enable the designer to make the most use of parallel computing technology. However, most of current batch approaches do not scale well with the batch size. That is, their performances deteriorate dramatically as the batch size increases. To address this issue, we propose a simple and efficient approach to extend Bayesian optimization to large-scale batch evaluation in this work. Different from existing batch approaches, the idea of the new approach is to draw a batch of axis-aligned subspaces of the original problem and select one acquisition point from each subspace. To achieve this, we propose the expected subspace improvement criterion to measure the amount of the improvement that a candidate point can achieve within a certain axis-aligned subspace. By optimizing these expected subspace improvement functions simultaneously, we can get a batch of query points for parallel evaluation. Numerical experiments show that our proposed approach can speedup the convergence significantly when compared with the sequential Bayesian optimization algorithm, and performs very competitively when compared with seven batch Bayesian optimization algorithms. A Matlab implementation of the proposed approach is available at https://github.com/zhandawei/Expected_Subspace_Improvement_Batch_Bayesian_Optimization.