Path Abstraction for Markov Reward Models

2026-08-25

AI Summary

本文扩展了路径抽象技术，从离散时间马尔可夫链的可达概率到马尔可夫奖励模型的期望奖励，并通过解线性方程组的方法实现了该技术。

Abstract

Path abstraction originated as a technique for counterexample refinement in probabilistic model checking. Given a discrete-time Markov chain, it summarises the probabilities passing through a subset of the states onto new transitions of a smaller chain. In earlier work, we proved its correctness and that it is monotonically absorbing. In this paper, we extend path abstraction from reachability probabilities on discrete-time Markov chains to expected rewards on Markov reward models. Working in a novel free monoid view of Markov chains throughout, we prove that path abstraction preserves the Markov reward model structure when abstracting over arbitrary sets of states, and that it remains monotonically absorbing. Finally, we give a numerical recipe, accompanied by a reference implementation in PARI/GP, that computes path abstraction by solving linear equation systems. Its correctness rests on the relationship between expected rewards and expected visiting times of transitions.

Problem

• path abstraction
• Markov reward models
• expected rewards
• monotonically absorbing

Innovation

• Path Abstraction
• Markov Reward Models
• Free Monoid View
• Monotonically Absorbing