Perception-Based Beliefs for POMDPs with Visual Observations

2026-02-05

AI Summary

This work addresses the challenge of partial observability in partially observable Markov decision processes (POMDPs) under high-dimensional visual observations, where traditional solvers struggle to scale effectively. The authors propose the Perception-Driven Belief Propagation (PBP) framework, which integrates an image classifier with a classical POMDP solver for the first time. The classifier maps raw images to state probability distributions, which are then incorporated into the belief update process, thereby circumventing direct handling of the high-dimensional observation space. To account for inaccuracies in the perception model, PBP further introduces an uncertainty quantification mechanism. Experimental results demonstrate that PBP outperforms end-to-end deep reinforcement learning approaches across multiple tasks, and that the explicit modeling of perceptual uncertainty significantly enhances robustness against visual disturbances.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Partially observable Markov decision processes (POMDPs) are a principled planning model for sequential decision-making under uncertainty. Yet, real-world problems with high-dimensional observations, such as camera images, remain intractable for traditional belief- and filtering-based solvers. To tackle this problem, we introduce the Perception-based Beliefs for POMDPs framework (PBP), which complements such solvers with a perception model. This model takes the form of an image classifier which maps visual observations to probability distributions over states. PBP incorporates these distributions directly into belief updates, so the underlying solver does not need to reason explicitly over high-dimensional observation spaces. We show that the belief update of PBP coincides with the standard belief update if the image classifier is exact. Moreover, to handle classifier imprecision, we incorporate uncertainty quantification and introduce two methods to adjust the belief update accordingly. We implement PBP using two traditional POMDP solvers and empirically show that (1) it outperforms existing end-to-end deep RL methods and (2) uncertainty quantification improves robustness of PBP against visual corruption.

Problem

• POMDPs
• visual observations
• belief update
• high-dimensional observations
• perception

Innovation

• Perception-based Beliefs
• POMDP
• visual observations
• uncertainty quantification
• belief update