PAC Learning in Turn-Based Stochastic Games with Reachability Objectives: A Decentralized Private Approach via Expected Conditional Distance

2026-07-16

AI Summary

This work addresses the long-standing challenge of PAC learning for reachability objectives in turn-based stochastic games under a decentralized setting with private information and heterogeneous learning algorithms on both sides. We propose the first PAC learning approach that enables collaborative learning using only local observations, grounded in game-theoretic modeling and a generalization of the Expected Conditional Distance (ECD) parameter. By integrating PAC learning theory with ECD analysis, we derive a sample complexity upper bound that scales polynomially with the number of states, actions, the ECD parameter, and the inverses of the error and failure probability. This result rigorously establishes the feasibility of PAC learning for reachability objectives in this challenging decentralized regime.

Abstract

Reachability is the most fundamental logical objective, yet it is notoriously difficult to learn in reinforcement learning settings: even for Markov decision processes, PAC learning of reachability is impossible without additional assumptions. This difficulty also holds in turn-based stochastic games (TBSGs), where two adversarial players interact on a finite state space. In this work, we consider turn-based stochastic games with reachability objectives. For such settings, adversarial learning, in which players are adversarial even in the learning phase, is impossible. Therefore, the goal is to consider learning, in which both players learn the unknown model together. In this spirit, previous literature on PAC learning in TBSGs considers (a)~public information shared by both players; and (b)~centralized learning, which means that players share the same learning algorithm. In this work, our contribution is two-fold. First, we relax these strong assumptions and ensure learning: (i)~with private information not shared with the other player; and (ii)~decentralized learning where the players do not share the same learning algorithm. To the best of our knowledge, this work is the first positive result for decentralized and private information learning of TBSGs with reachability objectives. Second, we introduce a game-theoretic generalization of the Expected Conditional Distance (ECD) parameter, which measures the expected length of reaching the target set. We establish a polynomial-sample complexity bound with respect to the number of states, actions, ECD parameter, and inverses of error tolerance and failure probability.

Problem

• PAC learning
• turn-based stochastic games
• reachability objectives
• decentralized learning
• private information

Innovation

• decentralized learning
• private information
• turn-based stochastic games
• reachability objectives
• Expected Conditional Distance