🤖 AI Summary
This work addresses the model checking problem of reachability probabilities for countably infinite-state Markov decision processes (MDPs). We first generalize the notion of *decisiveness*—originally defined for Markov chains—to MDPs, introducing two non-determinism-aware variants: *adversarial* and *cooperative decisiveness*. Based on these definitions, we establish sufficient conditions under which the infimum and supremum reachability probabilities are decidable. Leveraging these conditions, we devise an approximation algorithm that is both theoretically sound and computationally feasible. Our framework provides the first formal, qualitative-and-quantitative verification foundation for infinite-state MDPs, successfully handling canonical models such as random walks and concurrent systems. It advances probabilistic model checking by unifying decidability analysis with practical approximation, offering a methodological breakthrough in formal verification of stochastic infinite-state systems.
📝 Abstract
Decisiveness has proven to be an elegant concept for denumerable Markov chains: it is general enough to encompass several natural classes of denumerable Markov chains, and is a sufficient condition for simple qualitative and approximate quantitative model checking algorithms to exist. In this paper, we explore how to extend the notion of decisiveness to Markov decision processes. Compared to Markov chains, the extra non-determinism can be resolved in an adversarial or cooperative way, yielding two natural notions of decisiveness. We then explore whether these notions yield model checking procedures concerning the infimum and supremum probabilities of reachability properties.
