Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimension

2026-05-22

AI Summary

This work addresses the performance degradation of Bayesian optimization in high-dimensional settings caused by unknown effective dimensions. To overcome this challenge, we propose Dynamic Shared Embedding Bayesian Optimization (DSEBO), the first method that automatically adapts the embedding dimensionality without requiring prior knowledge. DSEBO initiates optimization in a low-dimensional subspace and dynamically increases the dimensionality based on convergence behavior, while efficiently reusing previously evaluated points through a shared embedding across successive subspaces. By integrating dynamic random embedding, a principled subspace switching mechanism, and theoretical analysis, DSEBO effectively balances approximation error and optimization error. Empirical results demonstrate that DSEBO significantly outperforms existing approaches across a range of high-dimensional benchmark functions and real-world tasks, achieving consistent improvements in both regret and computational efficiency.

Abstract

Bayesian optimization is widely employed for optimizing complex black-box functions but struggles with the curse of dimensionality. Random embedding, as a dimension reduction strategy, simplifies tasks that possess the effective dimension by optimizing within a low-dimensional subspace. However, determining the effective dimension of a task in advance remains a significant challenge, which influences the selection of the subspace dimensionality and the optimization performance. Traditional methods use fixed subspace dimensions provided by experts or rely on trial and error to estimate subspace dimensions with resources consumed. To this end, this paper proposes an automated random embedding for high-dimensional Bayesian optimization with unknown effective dimension, called Dynamic Shared Embedding Bayesian Optimization (DSEBO). DSEBO starts with a low dimension and switches to a higher subspace if the solutions in the current subspace show preliminary convergence. DSEBO dynamically determines the dimension of the next subspace based on the quality of the solutions in different subspaces and shares the queried solutions with the new subspace for a better initialization. Theoretically, we derive a regret bound for DSEBO and demonstrate that DSEBO can better balance approximation and optimization errors. Extensive experiments on functions with dimensionality of varying magnitudes and real-world tasks with unknown effective dimensions reveal that, compared with state-of-the-art methods, alternating optimization across different subspaces results in significant improvements in high-dimensional optimization, both in terms of optimization regret and time.

Problem

• Bayesian optimization
• random embedding
• effective dimension
• high-dimensional optimization
• subspace dimensionality

Innovation

• Bayesian optimization
• random embedding
• effective dimension
• dynamic subspace
• high-dimensional optimization