REBA: A Revealed Belief Automaton Framework for Online Planning in Continuous POMDPs

2026-06-20

AI Summary

This work addresses the challenge of generating reliable symbolic states online from continuous observations in continuous partially observable Markov decision processes (POMDPs) to support long-horizon reasoning under ω-regular specifications. The proposed REBA framework eschews predefined discrete abstractions and introduces, for the first time, an information-theoretic gating mechanism that dynamically identifies “revealing events” in an event-driven manner. This mechanism extracts trustworthy anchor points from the continuous belief space and incrementally constructs a finite belief automaton. By integrating ω-regular specifications to guide Monte Carlo tree search, REBA enables online planning beyond local horizons. Experimental results demonstrate that REBA substantially outperforms existing methods on patrol and navigation tasks, achieving performance improvements of 17.0%–47.4% on key metrics.

Abstract

Online planning in continuous partially observable Markov decision processes (POMDPs) using $ω$-regular specifications requires handling continuous belief dynamics within the finite symbolic memory in order to track temporal progress. Existing methods based on either direct search in belief space or predefined discrete abstractions suffer from drawbacks, e.g., lack of symbolic memory for long-horizon logical progress or difficult to certify from noisy online beliefs. As such, obtaining reliable symbolic states online from continuous observations remains a challenge. To address this issue, we introduce the Revealed Belief Automaton (REBA), an event-driven framework that advances the research from global belief-space discretization to a fundamental new way of thinking, namely online certification of revelation events. Specifically, we propose an online revelation method that, through information-theoretic gates, can dynamically analyse and establish belief abstraction from the continuous belief space by discovering reliable anchors among noisy beliefs. We then develop an incremental topology adaptation mechanism over the certified anchors to realise the online finite Belief Automaton. By combining with the $ω$-regular specification, REBA is able to support formal parity policy synthesis without a predefined discrete abstraction, which in turn can guide the Monte Carlo Tree Search process to perform online search beyond its local horizon. In addition, we design an error decomposition analysis which can assess the effectiveness and reliability of this discrete guidance for the underlying continuous POMDP. Empirical evaluations in patrolling and navigation scenarios show that REBA matches or exceeds all evaluated baselines, with primary metric gains of +17.0\% to +47.4\% over state-of-the-art approaches.

Problem

• continuous POMDPs
• online planning
• symbolic states
• belief abstraction
• ω-regular specifications

Innovation

• Revealed Belief Automaton
• online certification
• information-theoretic gates
• incremental topology adaptation
• ω-regular specifications