The impact of uncertainty on regularized learning in games

2025-06-16

AI Summary

This paper investigates the convergence behavior of Follow-the-Regularized-Leader (FTRL) dynamics in games under persistent stochastic perturbations. Addressing settings where payoff observations and strategy updates are continuously corrupted by noise, it identifies an “uncertainty-prefers-extremes” phenomenon: any non-degenerate noise drives the learning trajectory into an arbitrarily small neighborhood of a pure strategy in finite time. Theoretically, pure Nash equilibria are shown to be the only possible long-run limits; the set of stable pure strategies coincides precisely with those closed under the better-response mapping. This work provides the first rigorous characterization of pure-strategy convergence for stochastic FTRL dynamics, establishes finite-time upper bounds on convergence, uncovers boundary-drift mechanisms—such as the transition from cyclic orbits to boundary attraction in zero-sum games—and delivers necessary and sufficient conditions linking stability to game structure.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Social Networks and Social Media: Media and governance, opinion dynamics, filter bubbles, polarization

Abstract

In this paper, we investigate how randomness and uncertainty influence learning in games. Specifically, we examine a perturbed variant of the dynamics of"follow-the-regularized-leader"(FTRL), where the players' payoff observations and strategy updates are continually impacted by random shocks. Our findings reveal that, in a fairly precise sense,"uncertainty favors extremes": in any game, regardless of the noise level, every player's trajectory of play reaches an arbitrarily small neighborhood of a pure strategy in finite time (which we estimate). Moreover, even if the player does not ultimately settle at this strategy, they return arbitrarily close to some (possibly different) pure strategy infinitely often. This prompts the question of which sets of pure strategies emerge as robust predictions of learning under uncertainty. We show that (a) the only possible limits of the FTRL dynamics under uncertainty are pure Nash equilibria; and (b) a span of pure strategies is stable and attracting if and only if it is closed under better replies. Finally, we turn to games where the deterministic dynamics are recurrent - such as zero-sum games with interior equilibria - and we show that randomness disrupts this behavior, causing the stochastic dynamics to drift toward the boundary on average.

Problem

• How randomness affects learning in game dynamics
• Identifying pure strategies as outcomes under uncertainty
• Impact of noise on stability in recurrent games

Innovation

• Perturbed FTRL dynamics with random shocks
• Uncertainty drives convergence to pure strategies
• Randomness disrupts recurrent deterministic dynamics