The Adaptivity Barrier in Batched Nonparametric Bandits: Sharp Characterization of the Price of Unknown Margin

2025-11-05

AI Summary

This paper investigates the adaptivity cost in batched nonparametric contextual bandits when the smoothness parameter α is unknown, measured by “regret inflation”—the ratio between the regret of an adaptive algorithm and that of an oracle algorithm with known α. We establish, for the first time, that with a finite number of batches, unknown α necessarily induces polynomial regret inflation—a novel adaptivity barrier—which vanishes only when the number of batches exceeds log log T. To address this, we propose RoBIN: a novel algorithm that computes near-optimal batch allocation via convex optimization, coupled with adaptive binning and nonparametric regression. RoBIN achieves regret inflation within a polylogarithmic factor of the theoretical lower bound. Our results provide the first precise characterization of the fundamental limits of adapting to unknown smoothness under batch constraints.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We study batched nonparametric contextual bandits under a margin condition when the margin parameter $alpha$ is unknown. To capture the statistical cost of this ignorance, we introduce the regret inflation criterion, defined as the ratio between the regret of an adaptive algorithm and that of an oracle knowing $alpha$. We show that the optimal regret inflation grows polynomially with the horizon $T$, with exponent given by the value of a convex optimization problem that depends on the dimension, smoothness, and number of batches $M$. Moreover, the minimizer of this optimization problem directly prescribes the batch allocation and exploration strategy of a rate-optimal algorithm. Building on this principle, we develop RoBIN (RObust batched algorithm with adaptive BINning), which achieves the optimal regret inflation up to polylogarithmic factors. These results reveal a new adaptivity barrier: under batching, adaptation to an unknown margin parameter inevitably incurs a polynomial penalty, sharply characterized by a variational problem. Remarkably, this barrier vanishes once the number of batches exceeds order $log log T$; with only a doubly logarithmic number of updates, one can recover the oracle regret rate up to polylogarithmic factors.

Problem

• Characterizing adaptivity penalty for unknown margin in batched bandits
• Developing optimal algorithm achieving minimal regret inflation rate
• Identifying vanishing adaptivity barrier with doubly logarithmic batches

Innovation

• Developed RoBIN algorithm with adaptive binning
• Achieved optimal regret inflation via batch allocation
• Overcame adaptivity barrier with logarithmic batch updates