Learning Probabilistic Temporal Logic Specifications for Stochastic Systems

2025-05-17

AI Summary

This work addresses the problem of automatically learning Boolean combinations of probabilistic linear temporal logic (PLTL) formulas from positively and negatively labeled Markov chain trajectories, to characterize temporal behavioral differences in stochastic systems—such as reinforcement learning policies and probabilistic models. Methodologically, it introduces the first inductive PLTL formula learning framework, integrating context-free grammar-guided formula enumeration, semantic validation via PRISM-based probabilistic model checking, and Boolean set cover optimization to balance conciseness and interpretability. Compared to existing LTL learning approaches, the method achieves significant improvements in expressiveness, verifiability, and learning efficiency. Empirical evaluation across two representative stochastic system scenarios demonstrates its ability to efficiently extract accurate, compact, and formally verifiable PLTL specifications.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Machine Learning: Statistical Relational/Logic Learning
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

There has been substantial progress in the inference of formal behavioural specifications from sample trajectories, for example, using Linear Temporal Logic (LTL). However, these techniques cannot handle specifications that correctly characterise systems with stochastic behaviour, which occur commonly in reinforcement learning and formal verification. We consider the passive learning problem of inferring a Boolean combination of probabilistic LTL (PLTL) formulas from a set of Markov chains, classified as either positive or negative. We propose a novel learning algorithm that infers concise PLTL specifications, leveraging grammar-based enumeration, search heuristics, probabilistic model checking and Boolean set-cover procedures. We demonstrate the effectiveness of our algorithm in two use cases: learning from policies induced by RL algorithms and learning from variants of a probabilistic model. In both cases, our method automatically and efficiently extracts PLTL specifications that succinctly characterise the temporal differences between the policies or model variants.

Problem

• Infer probabilistic LTL formulas for stochastic systems
• Learn Boolean combinations from Markov chain classifications
• Extract concise temporal differences in RL policies

Innovation

• Grammar-based enumeration for concise PLTL formulas
• Probabilistic model checking for stochastic behavior
• Boolean set-cover procedures for specification inference