Sequential QCQP for Bilevel Optimization with Line Search

2025-05-20

AI Summary

This paper addresses core challenges in bilevel optimization—strong coupling between levels, difficulty in ensuring feasibility, and poor convergence. We propose a parameter-free single-loop algorithm. At each iteration, it solves a convex quadratically constrained quadratic program (QCQP) subproblem with a closed-form solution and introduces a backtracking line search inspired by controlled barrier functions—the first application of control barrier concepts to step-size selection. We establish an $O(1/k)$ ergodic convergence rate. Crucially, we design an *anytime* feasibility guarantee mechanism that requires no hyperparameter tuning. Experiments on standard bilevel tasks demonstrate that the algorithm simultaneously achieves strict descent in the upper-level objective and near-optimal satisfaction of the lower-level optimality condition. It exhibits high efficiency, robustness, and strong scalability across diverse problem instances.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Bilevel optimization involves a hierarchical structure where one problem is nested within another, leading to complex interdependencies between levels. We propose a single-loop, tuning-free algorithm that guarantees anytime feasibility, i.e., approximate satisfaction of the lower-level optimality condition, while ensuring descent of the upper-level objective. At each iteration, a convex quadratically-constrained quadratic program (QCQP) with a closed-form solution yields the search direction, followed by a backtracking line search inspired by control barrier functions to ensure safe, uniformly positive step sizes. The resulting method is scalable, requires no hyperparameter tuning, and converges under mild local regularity assumptions. We establish an O(1/k) ergodic convergence rate and demonstrate the algorithm's effectiveness on representative bilevel tasks.

Problem

• Solves bilevel optimization with hierarchical interdependencies
• Ensures anytime feasibility and upper-level objective descent
• Uses QCQP and line search for scalable convergence

Innovation

• Single-loop tuning-free bilevel optimization algorithm
• Convex QCQP with closed-form solution direction
• Backtracking line search for safe step sizes