Probabilistic Alternating Simulations for Policy Synthesis in Uncertain Stochastic Dynamical Systems

2025-08-07

AI Summary

Traditional probabilistic simulation-based policy synthesis methods lack expressive power for stochastic dynamical systems subject to both stochastic and set-valued nondeterministic disturbances. To address this, we propose **probabilistic alternating simulation relations**, the first formal framework unifying probabilistic and alternating simulation. This relation provides a rigorous behavioral abstraction foundation for systems with dual uncertainties—stochasticity and nondeterminism—enabling sound and complete policy synthesis over finite-state Markov decision process (MDP) abstractions while preserving both controllability and robustness. We validate our approach on a 4-dimensional Dubins vehicle model, demonstrating substantial improvements in control policy reliability and formal verifiability. The method extends the theoretical boundaries of formal verification and synthesis for uncertain systems, bridging a critical gap between probabilistic and game-theoretic abstractions in control design.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Multiagent Systems: Multiagent Systems under Uncertainty
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Novel mobile and WoT systems, and system-of-systems

Abstract

A classical approach to formal policy synthesis in stochastic dynamical systems is to construct a finite-state abstraction, often represented as a Markov decision process (MDP). The correctness of these approaches hinges on a behavioural relation between the dynamical system and its abstraction, such as a probabilistic simulation relation. However, probabilistic simulation relations do not suffice when the system dynamics are, next to being stochastic, also subject to nondeterministic (i.e., set-valued) disturbances. In this work, we extend probabilistic simulation relations to systems with both stochastic and nondeterministic disturbances. Our relation, which is inspired by a notion of alternating simulation, generalises existing relations used for verification and policy synthesis used in several works. Intuitively, our relation allows reasoning probabilistically over stochastic uncertainty, while reasoning robustly (i.e., adversarially) over nondeterministic disturbances. We experimentally demonstrate the applicability of our relations for policy synthesis in a 4D-state Dubins vehicle.

Problem

• Extends probabilistic simulation to stochastic and nondeterministic systems
• Generalizes relations for verification and policy synthesis
• Enables robust policy synthesis in uncertain dynamical systems

Innovation

• Extends probabilistic simulation to nondeterministic disturbances
• Combines stochastic and adversarial reasoning approaches
• Validated on 4D Dubins vehicle policy synthesis