Limit-sure reachability for small memory policies in POMDPs is NP-complete

2024-12-01

arXiv.org

AI Summary

This paper investigates the limit-sure reachability problem for partially observable Markov decision processes (POMDPs) under finite-memory policies: given a target state set, does there exist a finite-memory policy ensuring that the system reaches the target with probability arbitrarily close to one? We establish, for the first time, that this problem is NP-complete—resolving a longstanding gap in the computational complexity theory of POMDP reachability under small memory constraints. Our proof proceeds via a polynomial-time reduction from 3-SAT to the limit-sure reachability problem, coupled with a sound and complete verification algorithm that certifies membership in NP. This precise complexity characterization provides a fundamental theoretical boundary for the feasibility of lightweight POMDP controllers and offers direct guidance for formal policy synthesis in resource-constrained settings.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

A standard model that arises in several applications in sequential decision making is partially observable Markov decision processes (POMDPs) where a decision-making agent interacts with an uncertain environment. A basic objective in such POMDPs is the reachability objective, where given a target set of states, the goal is to eventually arrive at one of them. The limit-sure problem asks whether reachability can be ensured with probability arbitrarily close to 1. In general, the limit-sure reachability problem for POMDPs is undecidable. However, in many practical cases the most relevant question is the existence of policies with a small amount of memory. In this work, we study the limit-sure reachability problem for POMDPs with a fixed amount of memory. We establish that the computational complexity of the problem is NP-complete.

Problem

• Study limit-sure reachability in POMDPs with small memory
• Determine NP-completeness of fixed-memory POMDP reachability
• Analyze computational complexity for practical policy constraints

Innovation

• NP-complete limit-sure reachability in POMDPs
• Fixed small memory policies analysis
• Undecidable general case simplified