Multi-Objective Statistical Model Checking using Lightweight Strategy Sampling (extended version)

2025-11-17

AI Summary

Statistical reliability assessment of the Pareto front in multi-objective optimization remains challenging, particularly for non-deterministic systems requiring rigorous, verifiable guarantees. Method: This paper introduces the first lightweight statistical model checking framework supporting multi-objective Pareto queries. It integrates Monte Carlo simulation with incremental policy sampling, a progressive convergence mechanism, and three efficient heuristics to compute two-sided confidence intervals for the Pareto front under limited sampling budgets. The approach unifies statistical hypothesis testing with online learning, ensuring almost-sure convergence. Results: Evaluated on the Modest/modes platform, the method achieves high-fidelity approximation of the true Pareto front at significantly lower computational cost than existing approaches, while delivering statistically sound, verifiable confidence guarantees. It substantially enhances both the rigor and practicality of multi-attribute trade-off analysis in stochastic systems.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Statistical model checking delivers quantitative verification results with statistical guarantees by applying Monte Carlo simulation to formal models. It scales to model sizes and model types that are out of reach for exhaustive, analytical techniques. So far, it has been used to evaluate one property value at a time only. Many practical problems, however, require finding the Pareto front of optimal tradeoffs between multiple possibly conflicting optimisation objectives. In this paper, we present the first statistical model checking approach for such multi-objective Pareto queries, using lightweight strategy sampling to optimise over the model's nondeterministic choices. We first introduce an incremental scheme that almost surely converges to a statistically sound confidence band bounding the true Pareto front from both sides in the long run. To obtain a close underapproximation of the true front in finite time, we then propose three heuristic approaches that try to make the best of an a-priori fixed sampling budget. We implement our new techniques in the Modest Toolset's 'modes' simulator, and experimentally show their effectiveness on quantitative verification benchmarks.

Problem

• Statistical model checking optimizes multiple conflicting objectives simultaneously
• Lightweight sampling finds Pareto fronts for nondeterministic model choices
• Heuristic approaches approximate optimal tradeoffs within fixed sampling budgets

Innovation

• Lightweight strategy sampling for multi-objective optimization
• Incremental scheme converging to statistical confidence bands
• Three heuristic approaches for finite-time Pareto approximations