Bayesian updates from coalgebraic determinisation

2026-06-24

AI Summary

This work addresses the limitation of traditional POMDP determinization methods, which discard intermediate observations and thus fail to support full-history Bayesian updating. By integrating unifilarisation into a coalgebraic determinization framework, the authors introduce a support structure over a monoid that represents system states as prior distributions and defines transitions via Bayesian filtering. They establish, for the first time, that unifilarisation is a special case of coalgebraic determinization, thereby naturally embedding Bayesian updating within a categorical semantics. This approach extends to stochastic Mealy machines equipped with support structures, yielding a semantics finer than conventional Moore models. The resulting framework generates, for each input word, a family of output distributions satisfying causal constraints, making it well-suited for modeling reinforcement learning and sequential decision-making problems.

Abstract

The powerset construction is the classical determinisation procedure for nondeterministic finite automata. In the coalgebraic setting, this construction has been generalised to structured coalgebras, which are coalgebras equipped with extra data. For stochastic Moore machines over the distribution monad, a type of structured coalgebra, the determinisation construction induces a semantics assigning to each finite input word a distribution on the current output. This semantics is appropriate when only the current output matters, but it is too coarse for settings in which intermediate observations must also be taken into account, as is typical for agents solving POMDPs in control theory and reinforcement learning. In these contexts, agents need to condition on all realised observations, not just the final one, so to better plan for the future. This has been addressed from a category theoretic perspective through a procedure called ``unifilarisation'', which (in our context) takes a stochastic Mealy machine and produces a machine whose states are priors over the original state space and whose transitions are given by Bayesian filtering. Here we show that unifilarisation is an instance of coalgebraic determinisation. We work with Mealy machines over monads equipped with extra structure generalising the notion of the support of a distribution. We show that in this setting, unifilarisation arises from the general determinisation procedure. We then compare the resulting final coalgebra semantics with the Moore-style one. Instead of assigning only a distribution on current outputs to each finite input word, it yields causal stochastic behaviours, that is, families mapping input words to distributions on output words compatible with the ``causality'' constraint that outputs cannot depend on future inputs.

Problem

• POMDP
• Bayesian updating
• causal stochastic behaviours
• intermediate observations
• unifilarisation

Innovation

• coalgebraic determinisation
• unifilarisation
• Bayesian filtering
• causal stochastic behaviours
• structured coalgebras