How Sequential Algorithm Portfolios can benefit Black Box Optimization

2026-01-23

AI Summary

This work addresses the performance limitations of single algorithms in black-box optimization due to the absence of prior knowledge. It proposes a sequential portfolio strategy that dynamically allocates computational budget across multiple algorithms, leveraging their complementary strengths and the variance reduction effect observed on individual objective functions to enhance overall performance. Notably, the approach requires no parallelization—operating solely through sequential algorithm invocations—and consistently outperforms single-algorithm baselines while uncovering new potential in restart mechanisms and warm-start strategies. Extensive evaluation on the COCO platform using the BBOB benchmark suite across more than 200 algorithm combinations demonstrates that the proposed method reliably surpasses existing baselines, achieving an average relative performance improvement exceeding 14%.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In typical black-box optimization applications, the available computational budget is often allocated to a single algorithm, typically chosen based on user preference with limited knowledge about the problem at hand or according to some expert knowledge. However, we show that splitting the budget across several algorithms yield significantly better results. This approach benefits from both algorithm complementarity across diverse problems and variance reduction within individual functions, and shows that algorithm portfolios do NOT require parallel evaluation capabilities. To demonstrate the advantage of sequential algorithm portfolios, we apply it to the COCO data archive, using over 200 algorithms evaluated on the BBOB test suite. The proposed sequential portfolios consistently outperform single-algorithm baselines, achieving relative performance gains of over 14%, and offering new insights into restart mechanisms and potential for warm-started execution strategies.

Problem

• Black-box optimization
• Algorithm portfolios
• Computational budget allocation
• Sequential execution
• Performance improvement

Innovation

• Sequential Algorithm Portfolios
• Black-Box Optimization
• Algorithm Complementarity
• Variance Reduction
• COCO Benchmark