Thresholds for sensitive optimality and Blackwell optimality in stochastic games

2025-06-23

AI Summary

This paper investigates strategy refinement for two-player zero-sum perfect-information stochastic games under the average-payoff criterion, focusing on characterizing the Blackwell optimality threshold α_Bw and the d-sensitive optimality threshold α_d. Addressing the limitations of existing bounds—namely, their conservatism and high computational complexity—we derive, for the first time, an effective upper bound on α_d for all d > −1, and significantly improve the upper bound on α_Bw. Methodologically, we innovatively integrate algebraic tools—including separation bounds for algebraic numbers, Lagrange’s bound, Mahler’s measure, and multiplicity theorems—to rigorously analyze the range of discount factors over which optimal strategies remain invariant. Our theoretical results provide formal guarantees for efficiently computing Blackwell-optimal and d-sensitive-optimal strategies, substantially reducing algorithmic complexity and advancing the practical feasibility of long-run discounted analysis in algorithmic game theory.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Game Theory and Economic Paradigms: Cooperative Game Theory
• Search and Optimization: Adversarial Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We investigate refinements of the mean-payoff criterion in two-player zero-sum perfect-information stochastic games. A strategy is Blackwell optimal if it is optimal in the discounted game for all discount factors sufficiently close to $1$. The notion of $d$-sensitive optimality interpolates between mean-payoff optimality (corresponding to the case $d=-1$) and Blackwell optimality ($d=+infty$). The Blackwell threshold $α_{sf Bw} in [0,1[$ is the discount factor above which all optimal strategies in the discounted game are guaranteed to be Blackwell optimal. The $d$-sensitive threshold $α_{sf d} in [0,1[$ is defined analogously. Bounding $α_{sf Bw}$ and $α_{sf d}$ are fundamental problems in algorithmic game theory, since these thresholds control the complexity for computing Blackwell and $d$-sensitive optimal strategies, by reduction to discounted games which can be solved in $Oleft((1-α)^{-1} ight)$ iterations. We provide the first bounds on the $d$-sensitive threshold $α_{sf d}$ beyond the case $d=-1$, and we establish improved bounds for the Blackwell threshold $α_{sf Bw}$. This is achieved by leveraging separation bounds on algebraic numbers, relying on Lagrange bounds and more advanced techniques based on Mahler measures and multiplicity theorems.

Problem

• Study Blackwell and d-sensitive optimality thresholds in stochastic games
• Bound thresholds to compute optimal strategies efficiently
• Use algebraic number theory to improve threshold bounds

Innovation

• Blackwell optimality for discounted games
• d-sensitive optimality interpolation method
• algebraic number separation bounds