POMDP-based Object Search with Growing State Space and Hybrid Action Domain

2026-04-16

AI Summary

This study addresses the challenge of object search for mobile robots in complex indoor environments, where localization errors, limited fields of view, and occlusions hinder performance. The task is formulated as a high-dimensional partially observable Markov decision process (POMDP) with an expanding state space and a hybrid action space. To solve it efficiently online, the authors propose GNPF-kCT, a novel solver that integrates Monte Carlo tree search (MCTS), neural process networks, and k-center clustering. Key innovations include belief tree reuse, redundant action filtering, and an improved UCB criterion based on estimated belief diameter to enhance decision-making efficiency, along with a guessing strategy tailored for information-sparse scenarios under a grid-world model. Experiments demonstrate that Fetch and Stretch robots using GNPF-kCT significantly outperform existing POMDP-based, non-POMDP, and large language model (LLM) approaches in Gazebo simulations, with real-world tests in office environments further confirming its effectiveness and practicality.

Technology Category

• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Efficiently locating target objects in complex indoor environments with diverse furniture, such as shelves, tables, and beds, is a significant challenge for mobile robots. This difficulty arises from factors like localization errors, limited fields of view, and visual occlusion. We address this by framing the object-search task as a highdimensional Partially Observable Markov Decision Process (POMDP) with a growing state space and hybrid (continuous and discrete) action spaces in 3D environments. Based on a meticulously designed perception module, a novel online POMDP solver named the growing neural process filtered k-center clustering tree (GNPF-kCT) is proposed to tackle this problem. Optimal actions are selected using Monte Carlo Tree Search (MCTS) with belief tree reuse for growing state space, a neural process network to filter useless primitive actions, and k-center clustering hypersphere discretization for efficient refinement of high-dimensional action spaces. A modified upper-confidence bound (UCB), informed by belief differences and action value functions within cells of estimated diameters, guides MCTS expansion. Theoretical analysis validates the convergence and performance potential of our method. To address scenarios with limited information or rewards, we also introduce a guessed target object with a grid-world model as a key strategy to enhance search efficiency. Extensive Gazebo simulations with Fetch and Stretch robots demonstrate faster and more reliable target localization than POMDP-based baselines and state-of-the-art (SOTA) non-POMDP-based solvers, especially large language model (LLM) based methods, in object search under the same computational constraints and perception systems. Real-world tests in office environments confirm the practical applicability of our approach. Project page: https://sites.google.com/view/gnpfkct.

Problem

• object search
• mobile robots
• visual occlusion
• localization errors
• indoor environments

Innovation

• POMDP
• growing state space
• hybrid action space
• neural process
• k-center clustering