Online learning with Erdős-Rényi side-observation graphs

2026-04-28

AI Summary

This work addresses the adversarial multi-armed bandit problem under partial monitoring, where losses of unplayed actions are independently revealed with an unknown probability \( r \), corresponding to an Erdős–Rényi side-observation graph. The paper introduces the first adaptive algorithmic framework that achieves near-optimal regret bounds without prior knowledge of \( r \). Specifically, it proposes two algorithms: when \( r \geq \frac{\log T}{2N} \), the expected regret is \( O(\sqrt{(T/r)\log N}) \); for smaller \( r \), the regret bound becomes \( O(\sqrt{(T/r)\log(N+T)}) \). A fast estimation mechanism automatically identifies the regime of \( r \), enabling the framework to match the known-\( r \) lower bound up to logarithmic factors.

Abstract

We consider adversarial multi-armed bandit problems where the learner is allowed to observe losses of a number of arms beside the arm that it actually chose. We study the case where all non-chosen arms reveal their loss with a fixed but unknown probability $r$, independently of each other and the action of the learner. We propose two algorithms that work for different ranges of $r$. We show that after $T$ rounds in a bandit problem with $N$ arms, the expected regret of our first algorithm is $O(\sqrt{(T /r) \log N })$ whenever $r\ge(\log T)/(2N)$, while our second algorithm achieves a regret of $O(\sqrt{(T/r) \log (N+T)})$ for smaller values of $r$. We also give a quick estimation procedure that decides the range of~$r$. All our bounds are within logarithmic factors of the best achievable performance of any algorithm that is even allowed to know~$r$.

Problem

• online learning
• adversarial multi-armed bandits
• side observations
• Erdős-Rényi graphs
• regret minimization

Innovation

• online learning
• adversarial multi-armed bandits
• side observations
• Erdős-Rényi graph
• regret bound