Toward Optimal Second-Order Path-Length Guarantee for Adversarial Multi-Armed Bandits

2026-08-16

AI Summary

This study addresses the optimality of second-order path-length regret bounds in adversarial multi-armed bandits. Through rigorous theoretical analysis, we confirm the optimality potential of existing algorithms. To eliminate reliance on prior knowledge of path length, we propose an adaptively restarting mechanism with bounded increments alongside a novel path-length estimator. This approach achieves optimal second-order path-length regret bounds matching theoretical lower bounds without requiring parameter pre-specification. Consequently, this work not only completes the theoretical optimality proof for this problem but also provides a practical, prior-free adaptive solution that maintains rigorous theoretical guarantees.

Abstract

We study second-order path-length regret in adversarial $K$-armed bandits against oblivious loss sequences. Bubeck et al. [2019] designed an algorithm that achieves $\widetilde{\mathcal{O}}(K+\sqrt{KQ_{\infty,1}})$ regret, where $Q_{\infty,1}$ is the first-order path length, and left open whether $\widetilde{\mathcal{O}}(\text{poly}(K)\sqrt{1+Q_{\infty,2}})$ regret is achievable under bandit feedback, where $Q_{\infty,2}$ is the second-order path length. Somewhat surprisingly, we resolve this question positively by showing that with a more involved analysis, the exact same algorithm of Bubeck et al. [2019] achieves $\mathcal{O}\left(K\log(KT)+\sqrt{K\log(KT)\bigl(1+Q_{\infty,2}\bigr)}\right)$ expected regret when $Q_{\infty,2}$ is known, where $T$ is the horizon. This matches the $Ω(\sqrt{KQ_{\infty,2}})$ lower bound up to logarithmic factors and additive terms. We further remove the knowledge of $Q_{\infty,2}$ using an adaptive restart scheme whose path-length estimator has uniformly bounded increments.

Problem

• Adversarial Multi-Armed Bandits
• Second-Order Path-Length
• Regret Bound
• Oblivious Loss Sequences

Innovation

• Second-Order Path-Length
• Adversarial Multi-Armed Bandits
• Adaptive Restart Scheme
• Regret Analysis
• Path-Length Estimator