Thompson Sampling Is 2-Competitive for Mistakes

2026-07-14

AI Summary

This work investigates the performance of Thompson Sampling (TS) in Bayesian multi-armed bandits, measured by expected number of mistakes. Under the setting where arms possess independent latent variables and evolve only when selected, the paper establishes the first rigorous proof that TS achieves a competitive ratio of 2 against any policy—meaning its expected mistake count is at most twice that of the optimal policy. This result confirms a conjecture by Guha and Munagala (2014) and demonstrates that the factor of 2 is tight. The analysis leverages Bayesian inference, competitive ratio theory, and martingale techniques, and applies to arbitrary non-increasing sequence of round weights, including fixed time windows and geometric discounting. Consequently, the study provides strong theoretical guarantees for the near-optimality of TS across a broad range of practical scenarios.

Abstract

We consider Bayesian bandit models and prove that Thompson sampling makes at most twice the expected number of mistakes (selections of a suboptimal arm) as any other policy. Our analysis applies as long as the latent arm processes are independent and each arm evolves only when played. For stochastic bandits with best arm defined via mean reward, this confirms a conjecture of Guha and Munagala from 2014, where the factor $2$ is already best possible. The result holds under any nonincreasing sequence of round weights, including fixed horizon and geometric discounting.

Problem

• Thompson Sampling
• Bayesian bandits
• mistake bound
• multi-armed bandits
• suboptimal arm selection

Innovation

• Thompson Sampling
• Bayesian bandits
• mistake competitiveness
• suboptimal arm selection
• regret analysis