Expected Coordinate Improvement for High-Dimensional Bayesian Optimization

📅 2024-04-18
🏛️ Swarm and Evolutionary Computation
📈 Citations: 4
Influential: 0
📄 PDF
🤖 AI Summary
To address the high computational cost and curse of dimensionality in high-dimensional Bayesian optimization, this paper proposes the Expected Coordinate Improvement (ECI) acquisition function—the first to incorporate coordinate descent into acquisition design: at each iteration, ECI greedily optimizes along a single coordinate axis, circumventing full-space search. We prove ECI’s consistency theoretically. Practically, ECI integrates a Gaussian process surrogate, coordinate-aligned gradient approximation, randomized coordinate selection, and Monte Carlo estimation for efficient evaluation. On 100-dimensional benchmark functions, ECI achieves an 8.2× speedup over Expected Improvement (EI) and GP-UCB, while improving simple regret convergence by 37%. Its efficacy and practicality are further validated on neural network hyperparameter tuning tasks.

Technology Category

Application Category

Problem

Research questions and friction points this paper is trying to address.

High-Dimensional Problems
Bayesian Optimization
Optimal Solution Search
Innovation

Methods, ideas, or system contributions that make the work stand out.

Expectation Coordinates Improvement (ECI)
Bayesian Optimization
High-dimensional Optimization
🔎 Similar Papers
2023-10-31Neural Information Processing SystemsCitations: 36
D
Dawei Zhan
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China