Multi-Environment POMDPs with Finite-Horizon Objectives

2026-05-08

AI Summary

This work addresses the problem of efficiently computing optimal values and policies in finite-horizon partially observable Markov decision processes (POMDPs) with multiple environment models, where the initial state is adversarially chosen. We first establish that this problem remains PSPACE-complete even in the multi-environment setting. Subsequently, we propose the first practical and computationally efficient algorithm that integrates explicit modeling of adversarial initial states with finite-horizon dynamic programming. Empirical evaluation on several standard benchmarks demonstrates that our approach significantly outperforms the only previously known algorithm, thereby confirming its effectiveness and practical utility.

Abstract

Partially Observable Markov Decision Processes (POMDPs) are systems in which one agent interacts with a stochastic environment, and receives only partial information about the current state. In a multi-environment POMDP (MEPOMDP), the initial state is unknown, and assumed to be adversarially chosen. In this work we focus on computing the optimal value and policy in MEPOMDPs with finite-horizon objectives. That problem is known to be PSPACE-complete in POMDPs. Our main results are as follows: (1) we establish that it is also PSPACE-complete in the more general setting of MEPOMDPs; (2) we present a practical algorithm and evaluate it on classical benchmarks, significantly outperforming the only previously known algorithm.

Problem

• POMDP
• MEPOMDP
• finite-horizon
• partial observability
• adversarial initialization

Innovation

• MEPOMDP
• finite-horizon objectives
• PSPACE-completeness
• practical algorithm
• partial observability