Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation

2026-07-15

AI Summary

This work addresses the contextual combinatorial semi-bandit (CCSB) problem, where at each round only contextual information is observed, and the learner must select a combinatorial action satisfying a cardinality constraint to maximize cumulative reward, without assuming any structural properties of the action space. To tackle this setting, the authors propose SquareCB.Comb, an algorithm that balances exploration and exploitation by solving a convex optimization problem at each round and efficiently samples combinatorial actions. This method achieves, for the first time under general function approximation and arbitrary combinatorial action structures, a minimax-optimal regret bound of $O(\sqrt{m A T \log|\mathcal{F}|})$, without requiring additional assumptions on the action set. In the realizable setting, it matches the performance of the best existing policy search approaches while offering superior generalization capabilities.

Abstract

We study the contextual combinatorial semi-bandit (CCSB) problem with general reward function approximation. At each round, the learner observes a context, selects a combinatorial action consisting of a subset of basic arms, and receives the reward of each selected arm; the goal is to maximize the cumulative reward over time. We propose SquareCB.Comb, a computationally efficient algorithm that, at each round, solves a convex optimization problem to sample a combinatorial action that balances exploration and exploitation. SquareCB.Comb scales to large arm sets and imposes no structural assumptions on the action set beyond a cardinality bound of $m$ on each combinatorial action. We prove that SquareCB.Comb achieves a minimax optimal regret bound of $O(\sqrt{m A T \log |\mathcal{F}|})$, where $A$ is the number of arms, $m$ is the maximum number of arms in a combinatorial action, $T$ is the time horizon, and $\mathcal{F}$ is the reward function class. In the realizable setting, this bound matches the state-of-the-art regret guarantees achieved by policy search-based algorithms in the more restricted slate recommendation settings, while simultaneously generalizing to arbitrary combinatorial action structures and general reward function approximation.

Problem

• contextual combinatorial semi-bandits
• reward function approximation
• cumulative reward maximization
• combinatorial action
• minimax regret

Innovation

• contextual combinatorial semi-bandits
• general function approximation
• SquareCB.Comb
• minimax optimal regret
• convex optimization