Benefits and Costs of Adaptive Sampling

2026-04-27

AI Summary

This study investigates the mechanism by which adaptive sampling enhances the accuracy of arm mean estimation in multi-armed bandits and its trade-off with online experimentation cost. Focusing on minimizing mean squared error, theoretical analysis demonstrates that, under finite-sample regimes, adaptive Neyman allocation already outperforms uniform sampling. To exploit this advantage, the authors propose two strategies—SARP and NARP—that dynamically interpolate between inference accuracy and cumulative regret, achieving joint optimization while preserving optimal asymptotic convergence rates. Through rigorous theoretical analysis and extensive simulations, the proposed methods consistently improve estimation precision across diverse scenarios, effectively control performance loss, and converge to the full-information benchmark at the optimal rate.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Sampling/Simulation-based Search
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multi-armed bandits are widely used for sequential experimentation in clinical trials, recommendation systems, and online platforms. While regret minimization and valid inference from adaptively collected data have each been studied extensively, a basic question remains: when does adaptivity \emph{improve estimation precision} relative to uniform designs, and how should inference be balanced against the online cost of experimentation? We first study arm-level mean estimation under mean-squared-error (MSE) objectives. We characterize when an adaptive Neyman allocation, which allocates samples according to arm variance, yields strict MSE improvements over uniform sampling. When there is variance heterogeneity across arms, these improvements arise at modest sample sizes, clarifying that adaptivity can be preferable for inference not only asymptotically, but also in many practical finite-sample settings. We then study a joint inference-regret objective that accounts for the cost of assigning units to inferior arms during experimentation. We propose the Static-Allocation Rate Policy (SARP) and Neyman-Adaptive Rate Policy (NARP), which interpolates between inference- and regret-oriented policies by adjusting exploration to the local structure of the instance. We show that SARP and NARP converge to the complete-information benchmark at the optimal rate as the sampling budget grows. Our proposed policies are practically attractive as it linearly interpolates between any standard regret-minimizing algorithm and inference-targeting adaptive policies. Yet we show it still enjoys the oracle-based asymptotic optimal rate. Simulations support the theory by demonstrating improved precision over uniform allocation while controlling performance loss across a range of instances.

Problem

• adaptive sampling
• estimation precision
• multi-armed bandits
• inference-regret tradeoff
• variance heterogeneity

Innovation

• adaptive sampling
• multi-armed bandits
• Neyman allocation
• inference-regret tradeoff
• mean-squared error