Fixed Point Certificates for Reachability and Expected Rewards in MDPs

2025-01-20

AI Summary

Trustworthy verification of quantitative reachability and expected reward properties in Markov decision processes (MDPs) remains challenging due to the lack of lightweight, independently verifiable certificates. Method: This paper introduces a lightweight, formally verifiable certificate framework for MDP quantitative properties, grounded in fixed-point theory—the first systematic application of fixed-point semantics to certificate construction for arbitrary finite MDPs without structural restrictions or algorithmic coupling. The framework incurs low computational overhead and is fully formalized end-to-end in Isabelle/HOL. Contribution/Results: We integrate the framework with the Storm model checker for automated certificate generation and independent validation. We present the first formal certification on the Quantitative Verification Benchmark Set and release the complete, open-source toolchain. Our approach establishes the first certificate paradigm for probabilistic model checking that simultaneously achieves universality (applicable to all finite MDPs), formal verifiability (machine-checked correctness), and practicality (efficient generation and validation).

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

The possibility of errors in human-engineered formal verification software, such as model checkers, poses a serious threat to the purpose of these tools. An established approach to mitigate this problem are certificates -- lightweight, easy-to-check proofs of the verification results. In this paper, we develop novel certificates for model checking of Markov decision processes (MDPs) with quantitative reachability and expected reward properties. Our approach is conceptually simple and relies almost exclusively on elementary fixed point theory. Our certificates work for arbitrary finite MDPs and can be readily computed with little overhead using standard algorithms. We formalize the soundness of our certificates in Isabelle/HOL and provide a formally verified certificate checker. Moreover, we augment existing algorithms in the probabilistic model checker Storm with the ability to produce certificates and demonstrate practical applicability by conducting the first formal certification of the reference results in the Quantitative Verification Benchmark Set.

Problem

• MDPs
• Software Verification
• Model Checking

Innovation

• Markov Decision Processes
• Certificate Generation
• Software Verification