Information-Theoretic Meta Dynamic Programming for Signalling and Control of POMDPs

2026-06-16

AI Summary

This work addresses the challenge of coupled signaling and control in partially observable Markov decision processes (POMDPs) by proposing an information-theoretic meta-dynamic programming framework under average-cost constraints. By introducing a dual-coupled information state—comprising the posterior distribution of the system state and the distribution over this posterior as sufficient statistics—the optimal stochastic control policy is decomposed into a separated structure dependent solely on these two information states, and necessary and sufficient conditions for optimality are established. The approach integrates directed information measures, Bayesian recursions, and dynamic programming over the probability simplex, naturally reducing to the classical POMDP solution in the absence of communication. This framework provides the first analytically tractable optimal control solution for POMDPs with endogenous information constraints, laying a theoretical foundation for integrated communication-control intelligent systems.

Abstract

In this paper, we study the information-theoretic characterization of simultaneous signalling and control over channels modeled by partially observable Markov decision processes (POMDPs). The problem is formulated as an optimization over randomized control strategies that maximize the directed information from actions to observations, subject to an average-cost constraint. We derive a novel dynamic programming framework in which the state is defined on the space of conditional probability distributions, leading to a high-level ``meta'' dynamic program. Specifically, we show that two coupled information states, namely, the posterior distribution of the system state and a distribution over such posteriors, satisfy Markov recursions and provide sufficient statistics for optimal control. This structure enables the decomposition of optimal strategies into separated randomized policies that depend only on these information states. Our results establish necessary and sufficient conditions for optimality and unify classical stochastic control and information-theoretic formulations. In particular, we show that in the absence of signalling, the proposed framework reduces to the standard dynamic programming equations for POMDPs. The developed approach provides a principled foundation for analyzing and designing control systems with intrinsic information constraints.

Problem

• signalling
• control
• POMDPs
• directed information
• information-theoretic

Innovation

• information-theoretic control
• meta dynamic programming
• POMDPs
• directed information
• information states