Observer-Aware Probabilistic Planning Under Partial Observability

2025-02-14

AI Summary

This work addresses the problem of active information regulation planning for agents operating in partially observable environments while aware of an observer. To overcome the limitation of existing observer-aware models—which assume full observability—we propose the first Partially Observable Observer-Aware Markov Decision Process (PO-OAMDP), capable of handling dynamic hidden states and runtime goal changes. Methodologically, we integrate Partially Observable MDP (POMDP) modeling with Heuristic Search Value Iteration (HSVI), augmented by a customized initialization strategy to accelerate convergence. We theoretically analyze structural properties of PO-OAMDP and empirically demonstrate that our framework significantly improves policy legibility and predictability—key desiderata for explainable AI—while verifying enhanced HSVI convergence on benchmark tasks. The core contribution is the principled extension of observer-aware decision-making to partial observability, enabling controllable, interpretation-aware information disclosure.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this article, we are interested in planning problems where the agent is aware of the presence of an observer, and where this observer is in a partial observability situation. The agent has to choose its strategy so as to optimize the information transmitted by observations. Building on observer-aware Markov decision processes (OAMDPs), we propose a framework to handle this type of problems and thus formalize properties such as legibility, explicability and predictability. This extension of OAMDPs to partial observability can not only handle more realistic problems, but also permits considering dynamic hidden variables of interest. These dynamic target variables allow, for instance, working with predictability, or with legibility problems where the goal might change during execution. We discuss theoretical properties of PO-OAMDPs and, experimenting with benchmark problems, we analyze HSVI's convergence behavior with dedicated initializations and study the resulting strategies.

Problem

• Observer-aware planning under partial observability
• Optimizing information transmission via observations
• Extending OAMDPs for dynamic hidden variables

Innovation

• Observer-aware Markov decision processes
• Handles partial observability
• Dynamic hidden variables consideration