Efficient learning by implicit exploration in bandit problems with side observations

2026-04-27

AI Summary

This work investigates a partial feedback setting in online learning that lies between full information and the multi-armed bandit, where selecting an action reveals not only its own loss but also the losses of additional actions. The paper proposes two novel algorithms based on implicit exploration: the first achieves near-optimal regret guarantees in general settings without requiring prior knowledge of the observation structure, while the second is tailored to combinatorial optimization problems, simultaneously ensuring computational efficiency and strong theoretical performance. Both algorithms improve upon existing approaches in both information-theoretic and computational terms, and they are the first to attain near-optimal regret in environments with unknown observation structures.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We consider online learning problems under a partial observability model capturing situations where the information conveyed to the learner is between full information and bandit feedback. In the simplest variant, we assume that in addition to its own loss, the learner also gets to observe losses of some other actions. The revealed losses depend on the learner's action and a directed observation system chosen by the environment. For this setting, we propose the first algorithm that enjoys near-optimal regret guarantees without having to know the observation system before selecting its actions. Along similar lines, we also define a new partial information setting that models online combinatorial optimization problems where the feedback received by the learner is between semi-bandit and full feedback. As the predictions of our first algorithm cannot be always computed efficiently in this setting, we propose another algorithm with similar properties and with the benefit of always being computationally efficient, at the price of a slightly more complicated tuning mechanism. Both algorithms rely on a novel exploration strategy called implicit exploration, which is shown to be more efficient both computationally and information-theoretically than previously studied exploration strategies for the problem.

Problem

• bandit problems
• partial observability
• side observations
• online learning
• implicit exploration

Innovation

• implicit exploration
• partial observability
• online learning
• bandit problems
• regret minimization