Stopping Criteria for Value Iteration on Concurrent Stochastic Reachability and Safety Games

2025-05-27

AI Summary

Value iteration (VI) for concurrent stochastic games (CSGs) lacks provable accuracy guarantees, undermining reliability in quantitative verification and strategy synthesis. Method: We propose the first bounded value iteration (BVI) framework for CSGs with rigorous error bounds. Our approach maintains synchronized upper and lower bound sequences on the value function at each iteration and employs interval analysis to derive a sound convergence criterion: termination upon achieving an interval width ≤ ε ensures the true value error is bounded by ε. Contribution/Results: Unlike heuristic ε-closeness stopping conditions, our method provides the first theoretically complete, precision-controllable framework for synthesizing optimal strategies for reachability and safety objectives in CSGs. Experimental evaluation demonstrates substantial improvements in strategy reliability and practicality, establishing a solid foundation for automated verification and control of stochastic multi-agent systems.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We consider two-player zero-sum concurrent stochastic games (CSGs) played on graphs with reachability and safety objectives. These include degenerate classes such as Markov decision processes or turn-based stochastic games, which can be solved by linear or quadratic programming; however, in practice, value iteration (VI) outperforms the other approaches and is the most implemented method. Similarly, for CSGs, this practical performance makes VI an attractive alternative to the standard theoretical solution via the existential theory of reals. VI starts with an under-approximation of the sought values for each state and iteratively updates them, traditionally terminating once two consecutive approximations are $epsilon$-close. However, this stopping criterion lacks guarantees on the precision of the approximation, which is the goal of this work. We provide bounded (a.k.a. interval) VI for CSGs: it complements standard VI with a converging sequence of over-approximations and terminates once the over- and under-approximations are $epsilon$-close.

Problem

• Develop stopping criteria for value iteration in concurrent stochastic games
• Ensure precision guarantees for value iteration approximations
• Combine under- and over-approximations for bounded value iteration

Innovation

• Uses value iteration for concurrent stochastic games
• Introduces bounded interval value iteration
• Combines over- and under-approximations for precision