Resilient Strategies for Stochastic Systems: How Much Does It Take to Break a Winning Strategy?

2026-02-27

AI Summary

This study investigates the robustness of agent policies under uncertain perturbations—such as action corruption caused by actuator failures—specifically examining the magnitude of disturbances under which a policy can still guarantee reachability or safety objectives. For both Markov decision processes and stochastic games, the work formally introduces the notion of policy resilience for the first time and establishes a comprehensive theoretical framework encompassing perturbation modeling, impact aggregation (under expected or worst-case semantics), and quantitative analysis (e.g., via frequency-based metrics). The paper delineates solvability boundaries under different aggregation mechanisms and extends the framework to stochastic games, thereby providing a rigorous foundation for designing highly reliable autonomous systems.

Technology Category

• Multiagent Systems: Multiagent Systems under Uncertainty
• Reasoning under Uncertainty: Stochastic Optimization
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We study the problem of resilient strategies in the presence of uncertainty. Resilient strategies enable an agent to make decisions that are robust against disturbances. In particular, we are interested in those disturbances that are able to flip a decision made by the agent. Such a disturbance may, for instance, occur when the intended action of the agent cannot be executed due to a malfunction of an actuator in the environment. In this work, we introduce the concept of resilience in the stochastic setting and present a comprehensive set of fundamental problems. Specifically, we discuss such problems for Markov decision processes with reachability and safety objectives, which also smoothly extend to stochastic games. To account for the stochastic setting, we provide various ways of aggregating the amounts of disturbances that may have occurred, for instance, in expectation or in the worst case. Moreover, to reason about infinite disturbances, we use quantitative measures, like their frequency of occurrence.

Problem

• resilient strategies
• stochastic systems
• disturbances
• Markov decision processes
• stochastic games

Innovation

• resilient strategies
• stochastic systems
• Markov decision processes
• disturbance aggregation
• quantitative resilience