Online combinatorial optimization with stochastic decision sets and adversarial losses

2026-04-28

AI Summary

This work addresses the challenge in online combinatorial optimization where action sets are stochastically and dynamically unavailable—due to events such as sensor failures or road closures—and losses are generated by an adversarial environment. Building upon the Follow-The-Perturbed-Leader framework, the paper introduces a novel “counting dormant time” loss estimation technique. This approach unifies treatment across full-information, semi-bandit, and a newly proposed constrained feedback settings, and for the first time delivers an efficient algorithm with significantly improved theoretical guarantees for sleeping bandits under stochastic availability. Theoretical analysis establishes regret bounds for each setting, and empirical results demonstrate that the proposed algorithm consistently outperforms existing methods across diverse scenarios, particularly excelling in stochastic-availability sleeping bandit problems.

Abstract

Most work on sequential learning assumes a fixed set of actions that are available all the time. However, in practice, actions can consist of picking subsets of readings from sensors that may break from time to time, road segments that can be blocked or goods that are out of stock. In this paper we study learning algorithms that are able to deal with stochastic availability of such unreliable composite actions. We propose and analyze algorithms based on the Follow-The-Perturbed-Leader prediction method for several learning settings differing in the feedback provided to the learner. Our algorithms rely on a novel loss estimation technique that we call Counting Asleep Times. We deliver regret bounds for our algorithms for the previously studied full information and (semi-)bandit settings, as well as a natural middle point between the two that we call the restricted information setting. A special consequence of our results is a significant improvement of the best known performance guarantees achieved by an efficient algorithm for the sleeping bandit problem with stochastic availability. Finally, we evaluate our algorithms empirically and show their improvement over the known approaches.

Problem

• online combinatorial optimization
• stochastic decision sets
• adversarial losses
• sleeping bandit
• sequential learning

Innovation

• stochastic decision sets
• adversarial losses
• Counting Asleep Times
• sleeping bandit
• online combinatorial optimization