BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH

2026-04-13

AI Summary

This work addresses the suboptimal querying behavior of meta-Bayesian optimization when test tasks exhibit low similarity to meta-training tasks, leading to a mismatch between training and testing conditions. To mitigate this issue, the authors propose a unified adaptive framework that dynamically integrates meta-learning with a lookahead strategy: it leverages meta-learned priors when they are informative, and automatically falls back to the lookahead mechanism otherwise. Implemented within a single architecture, this approach enables automatic policy selection, substantially enhancing the robustness and sample efficiency of Bayesian optimization in environments with uncertain task structures. Experimental results demonstrate that the proposed framework consistently matches or outperforms existing methods across diverse function optimization benchmarks, particularly excelling when test tasks deviate significantly from the meta-training distribution.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Bayesian optimization (BO) has for sequential optimization of expensive black-box functions demonstrated practicality and effectiveness in many real-world settings. Meta-Bayesian optimization (meta-BO) focuses on improving the sample efficiency of BO by making use of information from related tasks. Although meta-BO is sample-efficient when task structure transfers, poor alignment between meta-training and test tasks can cause suboptimal queries to be suggested during online optimization. To this end, we propose a simple meta-BO algorithm that utilizes related-task information when determined useful, falling back to lookahead otherwise, within a unified framework. We demonstrate competitiveness of our method with existing approaches on function optimization tasks, while retaining strong performance in low task-relatedness regimes where test tasks share limited structure with the meta-training set.

Problem

• Bayesian optimization
• meta-learning
• task relatedness
• black-box optimization
• sample efficiency

Innovation

• Bayesian optimization
• meta-learning
• lookahead
• task relatedness
• sample efficiency