🤖 AI Summary
This paper resolves the long-standing open problem of decidability for Good-for-MDPs (GFM) automata: given a nondeterministic Büchi automaton, determine whether it is GFM—i.e., whether it can reliably implement lightweight Büchi acceptance in MDP model checking and reinforcement learning. We establish the first decidability result for GFM, presenting an EXPTIME decision procedure and proving a matching PSPACE-hard lower bound, thereby settling EXPTIME-completeness. Our approach integrates game-theoretic modeling, simulation via alternating Turing machines, symbolic fixed-point computation, and parity game solving. This work fills a critical theoretical gap at the intersection of formal verification and reinforcement learning, providing a rigorous complexity characterization and algorithmic foundation for the construction, verification, and application of GFM automata.
📝 Abstract
Nondeterministic Good-for-MDP (GFM) automata are for MDP model checking and reinforcement learning what good-for-games automata are for reactive synthesis: a more compact alternative to deterministic automata that displays nondeterminism, but only so much that it can be resolved locally, such that a syntactic product can be analysed. GFM has recently been introduced as a property for reinforcement learning, where the simpler B""uchi acceptance conditions it allows to use is key. However, while there are classic and novel techniques to obtain automata that are GFM, there has not been a decision procedure for checking whether or not an automaton is GFM. We show that GFM-ness is decidable and provide an EXPTIME decision procedure as well as a PSPACE-hardness proof.