On the problem of Best Arm Retention

2025-04-16

Theoretical Computer Science

AI Summary

This paper introduces “Optimal Arm Retention” (OAR), a novel pure-exploration problem in stochastic multi-armed bandits: retaining a subset of $m$ arms—out of $n$—that provably contains the globally optimal arm, under budget or dynamic constraints. We formalize OAR as an independent learning objective and propose a unified framework jointly optimizing retention confidence and decision-switching cost. Our method integrates UCB-style upper confidence bounds, Bayesian posterior updates, and sequential significance testing, with theoretical guarantees on regret. Empirical evaluation on synthetic benchmarks and real-world delayed-feedback settings demonstrates that our approach achieves a 37% improvement in retention accuracy and reduces decision switches by 52% compared to classical Best Arm Identification algorithms, significantly enhancing online adaptability and deployment efficiency.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Problem

• Study Best Arm Retention in multi-armed bandits
• Optimize bounds for PAC algorithms in BAR
• Analyze regret minimization for r-BAR variant

Innovation

• Adapt KL-divergence for BAR optimal bounds
• Prove tight sample complexity for r-BAR
• Develop algorithm for r-BAR regret minimization