Pareto-Optimal Anytime Algorithms via Bayesian Racing

2026-03-09

AI Summary

This work addresses the challenge of comparing anytime algorithms under unknown computational budgets, where existing methods often rely on objective normalization, subjective interpretation, or yield inconsistent results. The authors propose PolarBear, a novel framework that uniquely integrates Pareto optimality with Bayesian racing. By employing a temporal Plackett–Luce ranking model, PolarBear performs Bayesian inference over algorithm performance rankings at each time step, eliminating the need for objective value normalization or knowledge of optimal solutions. The approach constructs an anytime Pareto set based on non-dominated relationships, enabling consistent cross-instance aggregation. Furthermore, it incorporates adaptive sampling, uncertainty calibration, and early elimination strategies to facilitate efficient and robust algorithm selection under arbitrary time constraints and user-specified risk preferences.

Technology Category

• Search and Optimization: Algorithm Configuration
• Machine Learning: Learning Preferences or Rankings
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Selecting an optimization algorithm requires comparing candidates across problem instances, but the computational budget for deployment is often unknown at benchmarking time. Current methods either collapse anytime performance into a scalar, require manual interpretation of plots, or produce conclusions that change when algorithms are added or removed. Moreover, methods based on raw objective values require normalization, which needs bounds or optima that are often unavailable and breaks coherent aggregation across instances. We propose a framework that formulates anytime algorithm comparison as Pareto optimization over time: an algorithm is non-dominated if no competitor beats it at every timepoint. By using rankings rather than objective values, our approach requires no bounds, no normalization, and aggregates coherently across arbitrary instance distributions without requiring known optima. We introduce PolarBear (Pareto-optimal anytime algorithms via Bayesian racing), a procedure that identifies the anytime Pareto set through adaptive sampling with calibrated uncertainty. Bayesian inference over a temporal Plackett-Luce ranking model provides posterior beliefs about pairwise dominance, enabling early elimination of confidently dominated algorithms. The output Pareto set together with the posterior supports downstream algorithm selection under arbitrary time preferences and risk profiles without additional experiments.

Problem

• anytime algorithms
• Pareto optimization
• algorithm selection
• Bayesian racing
• performance comparison

Innovation

• Pareto optimization
• anytime algorithms
• Bayesian racing
• Plackett-Luce model
• algorithm selection