Accelerating Black-Box Bilevel Optimization with Rank-Based Upper-Level Value Function Approximation

2026-04-03

AI Summary

This work addresses the high computational cost of black-box bilevel optimization, particularly in multimodal settings with strong variable interactions where solving nested problems is notoriously difficult. The authors propose an efficient framework that, for the first time, incorporates a ranking-based approximation of the upper-level value function into bilevel optimization. By leveraging the invariance of rank-based evolutionary algorithms to monotonic transformations, the method directly models the ordinal relationships of the upper-level objective, thereby circumventing the need to fully converge the lower-level optimization at every iteration. Using CMA-ES as the continuous optimizer, the approach constructs a rank-invariant surrogate model that substantially reduces computational overhead. Empirical results on standard benchmarks demonstrate superior performance and the ability to solve complex bilevel problems previously intractable to existing methods.

Abstract

Bilevel optimization is a field of significant theoretical and practical interest, yet solving such optimization problems remains challenging. Evolutionary methods have been employed to address these problems in the black-box setting; however, they incur high computational cost due to the nested nature of bilevel optimization. Although previous methods have attempted to reduce this cost through various heuristic techniques, such approaches limit versatility on challenging optimization landscapes, such as those with multimodality and significant interaction between upper- and lower-level decision variables. In this study, we propose an efficient framework that exploits the invariance of rank-based evolutionary algorithms to monotonic transformations, thereby reducing the computational burden of the lower-level optimization loop. Specifically, our method directly approximates the rankings of the upper-level value function, bypassing the need to run the lower-level optimizer until convergence for each upper-level iteration. We apply this framework to the setting where both levels are continuous, adopting CMA-ES as the optimizer. We demonstrate that our method achieves competitive performance on standard bilevel optimization benchmarks and can solve problems that are intractable with previously proposed methods, particularly those with multimodality and strong inter-variable interactions.

Problem

• bilevel optimization
• black-box optimization
• computational cost
• multimodality
• variable interaction

Innovation

• bilevel optimization
• rank-based approximation
• black-box optimization
• CMA-ES
• value function approximation