Optimal Regret for Single Index Bandits

2026-05-10

AI Summary

This work addresses the single-index bandit problem in high-dimensional contexts, where rewards are determined by an unknown one-dimensional projection of the context vector followed by an unknown non-monotonic reward function. The authors propose ZoomSIB-UCB, a two-stage algorithm that first estimates the projection direction using a normalized Stein estimator and then reduces the problem to a one-dimensional bandit, solved via a UCB strategy. This approach achieves, for the first time, the optimal regret bound for non-monotonic single-index bandits without requiring additional assumptions, thereby filling a key theoretical gap. The analysis establishes an upper regret bound of Õ(T^{2/3}) and proves a matching minimax lower bound of Ω̃(T^{2/3}). Empirical results further demonstrate the algorithm’s effectiveness and robustness.

Abstract

We study the $\textit{single-index bandit}$ problem, where rewards depend on an unknown one-dimensional projection of high-dimensional contexts through an unknown reward function. This model extends linear and generalized linear bandits to a nonparametric setting, and is particularly relevant when the reward function is not known in advance. While optimal regret guarantees are known for monotone reward functions, the general non-monotone case remains poorly understood, with the best known bound being $\tilde{\mathcal{O}}(T^{3/4})$ (under standard boundedness and Lipschitz assumptions on the reward function [Kang et al., 2025]). We close this gap by establishing the optimal regret for general single-index bandits. We propose a simple two-phase algorithm, namely, Zoomed Single Index Bandit with Upper Confidence Bound ($\texttt{ZoomSIB-UCB}$), that first estimates the projection direction via a normalized Stein estimator, and then reduces the problem to a one-dimensional bandit using discretization and finally use UCB. This approach achieves a regret of $\tilde{\mathcal{O}}(T^{2/3})$, and improves significantly upon prior work without any additional assumptions. We also prove a matching minimax lower bound of $\tildeΩ(T^{2/3})$, showing that the upper bound is essentially tight. Our upper and lower bounds together provide a sharp characterization of the regret in single-index bandits. Moreover, the empirical results further demonstrate the effectiveness and robustness of our approach.

Problem

• single-index bandits
• non-monotone reward function
• optimal regret
• minimax lower bound

Innovation

• single-index bandits
• optimal regret
• nonparametric bandits
• minimax lower bound
• ZoomSIB-UCB