Constant regret in general games via higher-order optimism

2026-09-03

AI Summary

本文提出了一种名为HOOD的学习算法，通过结合高阶预测器和熵正则化方法，在一般博弈中实现了O(N^3 log^2 K)的个体遗憾上界。

Abstract

We introduce an uncoupled learning algorithm which, when employed by all players of an arbitrary $N$-player normal form game with up to $K$ actions per player, guarantees $O(N^3\log^2 K)$ individual regret, uniformly over the horizon of play. The proposed algorithm - which we call higher-order optimism with discounting (HOOD) is a variant of optimistic follow-the-regularized-leader (OptFTRL) that combines a discounted $(N+1)$-th order predictor with entropic regularization over a suitable "lifting" of the game's strategy space. This combination of ingredients is purposefully designed to dampen large oscillations of the induced sequence of play in a controlled manner, removing in this way a key stumbling block of previous attempts to achieve constant regret in general games. Our approach bears several striking similarities to the concurrent - and completely independent - work of Liu, Farina, and Ozdaglar (arXiv:2608.31166), who very recently derived an $O(N^{21}\log^{4} K)$ regret bound through the use of higher-order optimism and an exponential moving average estimator.

Problem

• constant regret
• normal form game
• uncoupled learning algorithm

Innovation

• higher-order optimism
• discounting
• entropic regularization
• lifting of strategy space
• controlled damping