Conformal Predictive Monitoring for Multi-Modal Scenarios

2025-09-01

AI Summary

Existing quantitative predictive monitoring (QPM) methods struggle to distinguish operational modes in multimodal stochastic systems, leading to overly conservative prediction intervals and the absence of mode-specific satisfaction guarantees. This paper proposes GenQPM, a mode-aware QPM framework that integrates score-based diffusion models (to approximate system dynamics), a lightweight mode classifier, and conformal inference to construct statistically valid, mode-specific prediction intervals. Its core innovation lies in embedding mode identification directly into the prediction pipeline, enabling runtime quantification of how well dynamic system states satisfy temporal logic properties. Evaluated on navigation and autonomous driving tasks, GenQPM significantly improves the informativeness and practicality of prediction intervals: it reduces average interval width by 32% and coverage deviation by 58% compared to mode-agnostic baseline methods.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We consider the problem of quantitative predictive monitoring (QPM) of stochastic systems, i.e., predicting at runtime the degree of satisfaction of a desired temporal logic property from the current state of the system. Since computational efficiency is key to enable timely intervention against predicted violations, several state-of-the-art QPM approaches rely on fast machine-learning surrogates to provide prediction intervals for the satisfaction values, using conformal inference to offer statistical guarantees. However, these QPM methods suffer when the monitored agent exhibits multi-modal dynamics, whereby certain modes may yield high satisfaction values while others critically violate the property. Existing QPM methods are mode-agnostic and so would yield overly conservative and uninformative intervals that lack meaningful mode-specific satisfaction information. To address this problem, we present GenQPM, a method that leverages deep generative models, specifically score-based diffusion models, to reliably approximate the probabilistic and multi-modal system dynamics without requiring explicit model access. GenQPM employs a mode classifier to partition the predicted trajectories by dynamical mode. For each mode, we then apply conformal inference to produce statistically valid, mode-specific prediction intervals. We demonstrate the effectiveness of GenQPM on a benchmark of agent navigation and autonomous driving tasks, resulting in prediction intervals that are significantly more informative (less conservative) than mode-agnostic baselines.

Problem

• Predicting temporal logic satisfaction in stochastic systems
• Handling multi-modal dynamics causing conservative predictions
• Generating mode-specific prediction intervals with statistical guarantees

Innovation

• Leverages deep generative models for multi-modal dynamics
• Employs mode classifier to partition trajectories
• Applies conformal inference for mode-specific prediction intervals