Generalized Neyman Allocation for Locally Minimax Optimal Best-Arm Identification

2024-05-29

AI Summary

This paper investigates the minimax optimality of best-arm identification (BAI) under a fixed budget in the small-gap regime. Addressing the challenge that the mean reward gap between optimal and suboptimal arms is vanishingly small—rendering error probability inherently difficult to suppress—we propose the first asymptotically local minimax-optimal algorithm. Our method theoretically achieves constant-factor tight matching between upper and lower bounds on the misidentification probability in the small-gap regime. By generalizing Neyman allocation to the multi-armed setting, we resolve a long-standing open problem in BAI concerning minimax-optimal sampling allocation. Integrating asymptotic statistical inference, information-theoretic lower bounds, and adaptive sampling theory, we derive a tight error bound with an explicit constant term within the local minimax framework. Compared to Glynn & Juneja’s approach and classical BAI algorithms, our bound on misidentification probability is substantially tighter.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

This study investigates an asymptotically locally minimax optimal algorithm for fixed-budget best-arm identification (BAI). We propose the Generalized Neyman Allocation (GNA) algorithm and demonstrate that its worst-case upper bound on the probability of misidentifying the best arm aligns with the worst-case lower bound under the small-gap regime, where the gap between the expected outcomes of the best and suboptimal arms is small. Our lower and upper bounds are tight, matching exactly including constant terms within the small-gap regime. The GNA algorithm generalizes the Neyman allocation for two-armed bandits (Neyman, 1934; Kaufmann et al., 2016) and refines existing BAI algorithms, such as those proposed by Glynn&Juneja (2004). By proposing an asymptotically minimax optimal algorithm, we address the longstanding open issue in BAI (Kaufmann, 2020) and treatment choice (Kasy&Sautmann, 202) by restricting a class of distributions to the small-gap regimes.

Problem

• Optimal Selection
• Limited Budget
• Error Probability Reduction

Innovation

• Generalized Neyman Allocation
• Optimization under budget constraints
• Error probability reduction