Practical, Utilitarian Algorithm Configuration

2025-10-16

AI Summary

This paper addresses the gap between strong theoretical guarantees and weak empirical performance of utilitarian algorithm configuration. Methodologically, it proposes a systematic enhancement of the COUP framework: (1) integrating an empirical performance augmentation strategy that significantly improves configuration quality while preserving theoretical guarantees (e.g., utility lower bounds); (2) enabling flexible modeling of runtime preferences and user-defined utility functions; and (3) establishing a robustness analysis framework under utility perturbations. The key contribution is the first demonstration that utilitarian configuration achieves empirical performance on par with state-of-the-art heuristic configurators—such as SMAC and IRACE—across multiple benchmarks, while rigorously maintaining theoretical optimality guarantees. This bridges the long-standing divide between theoretical rigor and practical competitiveness in algorithm configuration.

Technology Category

• Search and Optimization: Algorithm Configuration
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Reasoning under Uncertainty: Decision/Utility Theory

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Utilitarian algorithm configuration identifies a parameter setting for a given algorithm that maximizes a user's utility. Utility functions offer a theoretically well-grounded approach to optimizing decision-making under uncertainty and are flexible enough to capture a user's preferences over algorithm runtimes (e.g., they can describe a sharp cutoff after which a solution is no longer required, a per-hour cost for compute, or diminishing returns from algorithms that take longer to run). COUP is a recently-introduced utilitarian algorithm configuration procedure which was designed mainly to offer strong theoretical guarantees about the quality of the configuration it returns, with less attention paid to its practical performance. This paper closes that gap, bringing theoretically-grounded, utilitarian algorithm configuration to the point where it is competitive with widely used, heuristic configuration procedures that offer no performance guarantees. We present a series of improvements to COUP that improve its empirical performance without degrading its theoretical guarantees and demonstrate their benefit experimentally. Using a case study, we also illustrate ways of exploring the robustness of a given solution to the algorithm selection problem to variations in the utility function.

Problem

• Improving practical performance of utilitarian algorithm configuration
• Bridging theoretical guarantees with empirical competitiveness
• Exploring solution robustness to utility function variations

Innovation

• Utilitarian algorithm configuration maximizes user utility
• COUP procedure enhanced for competitive practical performance
• Improved empirical performance without degrading theoretical guarantees