Tracking the Best Strategy in an Extensive-Form Game

2026-08-10

AI Summary

This work addresses the challenge of minimizing regret against arbitrarily switching mixed strategies of forgetful opponents in extensive-form games under bandit feedback. To this end, we propose the first online learning algorithm that simultaneously achieves low switching regret and high computational efficiency. By leveraging the game tree structure and incorporating an adaptive parameter mechanism, our method effectively controls the frequency of strategy switches while optimizing the regret bound. The algorithm attains a switching regret bound of Õ((1/ρ + ρK)√(HAT)) and operates with a per-round time complexity of only O(HB), significantly enhancing both scalability and practical applicability.

Abstract

We consider the extensive-form bandit problem where on each trial the learner plays an extensive-form game against an oblivious adversary. We focus on the notion of switching regret, which measures the expected performance of the learner against that of any switching sequence of mixed strategies in retrospect. Our algorithm takes a parameter $ρ>0$ and achieves a switching regret of $\tilde{\mathcal{O}}((1/ρ+ρK)\sqrt{H A T})$ where $K$ is the number of switches in the comparator sequence, $H$ is the maximum number of the learner's information sets that can be traversed during a play of the game and $A$ is the number of actions that the learner can possibly take. Our algorithm is extremely efficient, taking a per trial time of only $\mathcal{O}(H B)$ where $B$ is the maximum number of actions available to the learner at any of its information sets.

Problem

• extensive-form game
• bandit problem
• switching regret
• oblivious adversary
• online learning

Innovation

• extensive-form games
• switching regret
• bandit feedback
• efficient online learning
• adaptive strategies