Supermartingale Certificates for Parametric MDPs

2026-09-11

AI Summary

研究通过参数平滑转换和引入参数超鞅证书，解决了参数化马尔可夫决策过程中的验证与合成问题。

Abstract

We consider the problems of formal verification and synthesis in parametric Markov decision processes (MDPs) with general measurable state and action spaces. The heart of our approach is a parameter flattening transformation, which allows us to transform parametric MDPs into semantically equivalent non-parametric MDPs. Building on this transformation, we introduce the novel notion of parametric supermartingale certificates, which generalize the traditional supermartingale certificates---used for non-parametric MDPs---to the parametric setting. We use our parametric supermartingale certificates to design algorithms for verification and approximate synthesis in polynomial arithmetic parametric MDPs. This leads to the first verification and synthesis algorithms for parametric MDPs with general state and action spaces. We implement our algorithms and experimentally evaluate them on several continuous parametric random walk benchmarks.

Problem

• parametric MDPs
• formal verification
• synthesis
• general state and action spaces

Innovation

• parametric supermartingale certificates
• parameter flattening transformation
• verification and synthesis algorithms