An $\epsilon$-Optimal Sequential Approach for Solving zs-POSGs

2026-02-27

AI Summary

This work addresses the nonlinearities and exponential computational complexity arising from synchronous minimax backups in zero-sum partially observable stochastic games (zs-POSGs). To overcome these challenges, the authors propose a sequential decision-making reformulation grounded in the separation principle. By introducing sequential occupancy states and private occupancy families as sufficient statistics for value estimation and policy execution, the approach uncovers the underlying geometric structure of the optimal value function, thereby enabling linearization of the backup operator. This reformulation reduces policy update complexity from exponential to polynomial time and directly yields security-equivalent strategies. Experimental results demonstrate that the proposed algorithm significantly outperforms existing methods, achieving the first efficient solution to large-scale zs-POSG instances previously deemed intractable.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Adversarial Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

While recent reductions of zero-sum partially observable stochastic games (zs-POSGs) to transition-independent stochastic games (TI-SGs) theoretically admit dynamic programming, practical solutions remain stifled by the inherent non-linearity and exponential complexity of the simultaneous minimax backup. In this work, we surmount this computational barrier by rigorously recasting the simultaneous interaction as a sequential decision process via the principle of separation. We introduce distinct sufficient statistics for valuation and execution, the sequential occupancy state and the private occupancy family, which reveal a latent geometry in the optimal value function. This structural insight allows us to linearise the backup operator, reducing the update complexity from exponential to polynomial while enabling the direct extraction of safe policies without heuristic bookkeeping. Experimental results demonstrate that algorithms leveraging this sequential framework significantly outperform state-of-the-art methods, effectively rendering previously intractable domains solvable.

Problem

• zero-sum partially observable stochastic games
• simultaneous minimax backup
• exponential complexity
• non-linearity
• computational intractability

Innovation

• sequential decision process
• occupancy state
• value function linearization
• zero-sum POSGs
• polynomial complexity