🤖 AI Summary
Existing algorithms for stochastic linear bandits—such as Greedy, OFUL, and Thompson Sampling—exhibit strong empirical performance but suffer from pessimistic theoretical regret bounds (e.g., only Õ(d√T) in the worst case), revealing a gap between practice and theory.
Method: This paper proposes a data-driven dynamic calibration framework leveraging the geometric properties of uncertainty ellipsoids. It introduces an instance-dependent frequentist regret bound grounded in ellipsoidal geometry and designs a plug-and-play “curriculum correction” mechanism that adaptively rescales online confidence sets without modifying the base algorithm.
Contribution/Results: The calibrated algorithms achieve the minimax-optimal regret bound Õ(d√T) on *every* instance. Experiments on synthetic and real-world datasets demonstrate substantial improvements in robustness and empirical performance, while preserving the computational efficiency and simplicity of the original algorithms.
📝 Abstract
This paper is motivated by recent research in the $d$-dimensional stochastic linear bandit literature, which has revealed an unsettling discrepancy: algorithms like Thompson sampling and Greedy demonstrate promising empirical performance, yet this contrasts with their pessimistic theoretical regret bounds. The challenge arises from the fact that while these algorithms may perform poorly in certain problem instances, they generally excel in typical instances. To address this, we propose a new data-driven technique that tracks the geometric properties of the uncertainty ellipsoid around the main problem parameter. This methodology enables us to formulate an instance-dependent frequentist regret bound, which incorporates the geometric information, for a broad class of base algorithms, including Greedy, OFUL, and Thompson sampling. This result allows us to identify and ``course-correct"problem instances in which the base algorithms perform poorly. The course-corrected algorithms achieve the minimax optimal regret of order $ ilde{mathcal{O}}(dsqrt{T})$ for a $T$-period decision-making scenario, effectively maintaining the desirable attributes of the base algorithms, including their empirical efficacy. We present simulation results to validate our findings using synthetic and real data.
