Robust Permissive Controller Synthesis for Interval MDPs

2025-10-03

AI Summary

This paper addresses robust controller synthesis for robotic manipulation under dynamic uncertainties arising from perception noise and actuation errors, formalized as Interval Markov Decision Processes (IMDPs), where transition probabilities are known only within bounded intervals—not exact values—and controllers must satisfy reachability or reward-based formal specifications across all admissible transition models. Method: We propose the first robust, fault-tolerant controller synthesis framework for IMDPs, combining mixed-integer linear programming (MILP) modeling with a dual-based efficient encoding technique to circumvent the computational bottleneck of vertex enumeration inherent in traditional approaches. Contribution/Results: Our method synthesizes maximally permissive robust controllers on four benchmark robotic tasks, guarantees strict formal correctness under all interval-compatible dynamics, and scales to IMDPs with over 100,000 states—achieving both runtime efficiency and strong formal assurances.

Technology Category

• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

We address the problem of robust permissive controller synthesis for robots operating under uncertain dynamics, modeled as Interval Markov Decision Processes (IMDPs). IMDPs generalize standard MDPs by allowing transition probabilities to vary within intervals, capturing epistemic uncertainty from sensing noise, actuation imprecision, and coarse system abstractions-common in robotics. Traditional controller synthesis typically yields a single deterministic strategy, limiting adaptability. In contrast, permissive controllers (multi-strategies) allow multiple actions per state, enabling runtime flexibility and resilience. However, prior work on permissive controller synthesis generally assumes exact transition probabilities, which is unrealistic in many robotic applications. We present the first framework for robust permissive controller synthesis on IMDPs, guaranteeing that all strategies compliant with the synthesized multi-strategy satisfy reachability or reward-based specifications under all admissible transitions. We formulate the problem as mixed-integer linear programs (MILPs) and propose two encodings: a baseline vertex-enumeration method and a scalable duality-based method that avoids explicit enumeration. Experiments on four benchmark domains show that both methods synthesize robust, maximally permissive controllers and scale to large IMDPs with up to hundreds of thousands of states.

Problem

• Synthesizing robust permissive controllers for uncertain robotic dynamics
• Addressing epistemic uncertainty in IMDPs with interval transition probabilities
• Ensuring specification compliance under all admissible transition variations

Innovation

• Robust permissive controller synthesis for IMDPs
• Mixed-integer linear programs for problem formulation
• Scalable duality-based method for large IMDPs