Utilitarian Algorithm Configuration for Infinite Parameter Spaces

2024-05-28

arXiv.org

AI Summary

This work addresses the low search efficiency and lack of theoretical guarantees in automated algorithm configuration over continuous or infinite parameter spaces. We propose the first utilitarian configuration framework for infinite parameter spaces, along with the COUP algorithm. COUP integrates optimistic-estimate-driven continuous sampling, utility-oriented adaptive configuration, procrastination-style delayed sampling, and progressive confidence upper-bound analysis—ensuring convergence while substantially improving search efficiency. Theoretically, COUP achieves a sublinear regret bound and superior runtime complexity compared to existing methods. Empirically, on diverse algorithm configuration tasks—including SAT solvers and machine learning hyperparameter optimization—COUP achieves 2.3–5.7× speedup over state-of-the-art baselines, while yielding higher-quality configurations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Utilitarian algorithm configuration is a general-purpose technique for automatically searching the parameter space of a given algorithm to optimize its performance, as measured by a given utility function, on a given set of inputs. Recently introduced utilitarian configuration procedures offer optimality guarantees about the returned parameterization while provably adapting to the hardness of the underlying problem. However, the applicability of these approaches is severely limited by the fact that they only search a finite, relatively small set of parameters. They cannot effectively search the configuration space of algorithms with continuous or uncountable parameters. In this paper we introduce a new procedure, which we dub COUP (Continuous, Optimistic Utilitarian Procrastination). COUP is designed to search infinite parameter spaces efficiently to find good configurations quickly. Furthermore, COUP maintains the theoretical benefits of previous utilitarian configuration procedures when applied to finite parameter spaces but is significantly faster, both provably and experimentally.

Problem

• Optimize algorithm performance in infinite parameter spaces
• Search continuous parameters efficiently for good configurations
• Maintain theoretical benefits while improving speed significantly

Innovation

• Optimizes infinite parameter spaces
• Ensures optimality with efficiency
• Faster than finite space methods