Bayesian Optimization of Bilevel Problems

2024-12-24

AI Summary

For bilevel optimization problems where both the upper- and lower-level objectives are expensive black-box functions, this paper proposes the first systematic Bayesian optimization framework tailored to general bilevel black-box settings. The method jointly models the upper- and lower-level decision spaces using a Gaussian process surrogate, enabling cross-level knowledge transfer. It introduces a novel acquisition function specifically designed for bilevel structure, optimizing inner- and outer-level responses in a coordinated manner. By unifying the modeling of hierarchical dependencies and avoiding nested optimization, the approach significantly reduces function evaluation overhead. On multiple benchmark bilevel problems, the method achieves 35–62% improvement in sample efficiency and consistently outperforms existing black-box bilevel optimization methods in both convergence speed and final solution quality.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Bilevel optimization, a hierarchical mathematical framework where one optimization problem is nested within another, has emerged as a powerful tool for modeling complex decision-making processes in various fields such as economics, engineering, and machine learning. This paper focuses on bilevel optimization where both upper-level and lower-level functions are black boxes and expensive to evaluate. We propose a Bayesian Optimization framework that models the upper and lower-level functions as Gaussian processes over the combined space of upper and lower-level decisions, allowing us to exploit knowledge transfer between different sub-problems. Additionally, we propose a novel acquisition function for this model. Our experimental results demonstrate that the proposed algorithm is highly sample-efficient and outperforms existing methods in finding high-quality solutions.

Problem

• Bilevel Optimization
• Complexity
• Efficiency

Innovation

• Bayesian Optimization
• Complex Bilevel Optimization Problems
• Enhanced Performance Tool