LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization

2026-03-03

AI Summary

This work addresses the challenge in black-box optimization of balancing global exploration with rapid local convergence. To this end, the authors propose the LAGO framework, which synergistically combines gradient-enhanced Bayesian optimization and trust-region local optimization through an adaptive competition mechanism. At each iteration, LAGO generates candidate points in parallel from both global and local strategies and dynamically selects the evaluation point based on predicted improvement. A key innovation lies in decoupling global exploration and local refinement at the candidate generation stage, while introducing a length-scale-based minimum distance criterion to filter local data, thereby mitigating numerical instability and ensuring both global coverage and efficient local convergence. Experimental results demonstrate that LAGO significantly outperforms conventional nonlinear local optimization methods on smooth functions, achieving superior global exploration capability alongside fast local convergence.

Technology Category

• Search and Optimization: Local Search
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

We introduce LAGO, a LocAl-Global Optimization algorithm that combines gradient-enhanced Bayesian Optimization (BO) with gradient-based trust region local refinement through an adaptive competition mechanism. At each iteration, global and local optimization strategies independently propose candidate points, and the next evaluation is selected based on predicted improvement. LAGO separates global exploration from local refinement at the proposal level: the BO acquisition function is optimized outside the active trust region, while local function and gradient evaluations are incorporated into the global gradient-enhanced Gaussian process only when they satisfy a lengthscale-based minimum-distance criterion, reducing the risk of numerical instability during the local exploitation. This enables efficient local refinement when reaching promising regions, without sacrificing a global search of the design space. As a result, the method achieves an improved exploration of the full design space compared to standard non-linear local optimization algorithms for smooth functions, while maintaining fast local convergence in regions of interest.

Problem

• global optimization
• local refinement
• Bayesian optimization
• trust region methods
• numerical stability

Innovation

• Local-Global Optimization
• Trust Region Methods
• Bayesian Optimization
• Gradient-Enhanced Gaussian Process
• Adaptive Competition Mechanism