Best Arm Identification with Resource Constraints

2024-02-29

International Conference on Artificial Intelligence and Statistics

AI Summary

This paper studies Best-Arm Identification with Resource Constraints (BAIwRC), where arms consume heterogeneous resources upon pulling, and the objective is to identify the optimal arm with high probability within a fixed total resource budget. We establish the first theoretical framework for BAIwRC, revealing a fundamental distinction in convergence rates between deterministic and stochastic resource consumption. Building upon the Successive Halving paradigm, we propose the SH-RR algorithm, which integrates a resource-quota allocation mechanism and non-asymptotic analysis techniques to derive a near-optimal non-asymptotic success probability bound. Theoretically, SH-RR achieves a convergence rate matching the information-theoretic lower bound up to constant factors—significantly improving upon existing BAI methods. Empirical results demonstrate that SH-RR substantially enhances both identification success rate and robustness under limited resource budgets.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Online Learning & Bandits
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Motivated by the cost heterogeneity in experimentation across different alternatives, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem. The agent aims to identify the best arm under resource constraints, where resources are consumed for each arm pull. We make two novel contributions. We design and analyze the Successive Halving with Resource Rationing algorithm (SH-RR). The SH-RR achieves a near-optimal non-asymptotic rate of convergence in terms of the probability of successively identifying an optimal arm. Interestingly, we identify a difference in convergence rates between the cases of deterministic and stochastic resource consumption.

Problem

• Identify best arm under resource constraints
• Design Successive Halving with Resource Rationing algorithm
• Analyze convergence rates for resource consumption types

Innovation

• Successive Halving with Resource Rationing algorithm
• Near-optimal non-asymptotic convergence rate
• Analyzes deterministic vs stochastic resource consumption