Expected Coordinate Improvement for High-Dimensional Bayesian Optimization

๐Ÿ“… 2024-04-18
๐Ÿ›๏ธ Swarm and Evolutionary Computation
๐Ÿ“ˆ Citations: 4
โœจ Influential: 0
๐Ÿ“„ PDF
๐Ÿค– AI Summary
To address the high computational cost and curse of dimensionality in high-dimensional Bayesian optimization, this paper proposes the Expected Coordinate Improvement (ECI) acquisition functionโ€”the first to incorporate coordinate descent into acquisition design: at each iteration, ECI greedily optimizes along a single coordinate axis, circumventing full-space search. We prove ECIโ€™s consistency theoretically. Practically, ECI integrates a Gaussian process surrogate, coordinate-aligned gradient approximation, randomized coordinate selection, and Monte Carlo estimation for efficient evaluation. On 100-dimensional benchmark functions, ECI achieves an 8.2ร— speedup over Expected Improvement (EI) and GP-UCB, while improving simple regret convergence by 37%. Its efficacy and practicality are further validated on neural network hyperparameter tuning tasks.

Technology Category

Application Category

Problem

Research questions and friction points this paper is trying to address.

High-Dimensional Problems
Bayesian Optimization
Optimal Solution Search
Innovation

Methods, ideas, or system contributions that make the work stand out.

Expectation Coordinates Improvement (ECI)
Bayesian Optimization
High-dimensional Optimization
๐Ÿ”Ž Similar Papers
2023-10-31Neural Information Processing SystemsCitations: 36