Value-Set Iteration: Computing Optimal Correlated Equilibria in Infinite-Horizon Multi-Player Stochastic Games

2025-06-08

AI Summary

This paper addresses the efficient computation of history-dependent optimal correlated equilibria (CE) in infinite-horizon multi-player stochastic games. Due to exponential growth of possible histories over time, standard approaches become intractable. To overcome this, we introduce a fault-tolerant (ε,δ)-optimal CE notion and propose—novelly—the compact representation of inducible value sets, coupled with a value-set iteration algorithm that models CE as the greatest fixed point of an update operator. Our method achieves the first polynomial-time computation: complexity is O((1/(εδ(1−γ)))^{n+1}) for general stochastic games, and reduces to polynomial in the number of agents n for turn-based variants. We further establish matching hardness results, proving that no algorithm can achieve better bi-criteria approximation under standard complexity assumptions. This work breaks long-standing representational and computational bottlenecks for history-dependent CE in stochastic games.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Game Theory and Economic Paradigms: Cooperative Game Theory
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

We study the problem of computing optimal correlated equilibria (CEs) in infinite-horizon multi-player stochastic games, where correlation signals are provided over time. In this setting, optimal CEs require history-dependent policies; this poses new representational and algorithmic challenges as the number of possible histories grows exponentially with the number of time steps. We focus on computing $(epsilon, delta)$-optimal CEs -- solutions that achieve a value within $epsilon$ of an optimal CE, while allowing the agents' incentive constraints to be violated by at most $delta$. Our main result is an algorithm that computes an $(epsilon,delta)$-optimal CE in time polynomial in $1/(epsilondelta(1 - gamma))^{n+1}$, where $gamma$ is the discount factor, and $n$ is the number of agents. For (a slightly more general variant of) turn-based games, we further reduce the complexity to a polynomial in $n$. We also establish that the bi-criterion approximation is necessary by proving matching inapproximability bounds. Our technical core is a novel approach based on inducible value sets, which leverages a compact representation of history-dependent CEs through the values they induce to overcome the representational challenge. We develop the value-set iteration algorithm -- which operates by iteratively updating estimates of inducible value sets -- and characterize CEs as the greatest fixed point of the update map. Our algorithm provides a groundwork for computing optimal CEs in general multi-player stochastic settings.

Problem

• Computing optimal correlated equilibria in infinite-horizon stochastic games
• Overcoming exponential history growth in multi-player game policies
• Achieving efficient bi-criterion approximation for incentive constraints

Innovation

• Value-set iteration for optimal correlated equilibria
• Polynomial-time algorithm for (ε, δ)-optimal CEs
• Inducible value sets compactly represent history-dependent CEs