Hierarchical Object-Oriented POMDP Planning for Object Rearrangement

2024-12-02

AI Summary

This work addresses the multi-object rearrangement task in partially observable, multi-room environments. We propose HOO-POMDP, a hierarchical planning framework that employs object-oriented state abstraction and online subgoal generation at the high level, while coupling pretrained policy networks for modular execution at the low level. To our knowledge, this is the first approach to integrate object-oriented modeling into hierarchical POMDPs, unifying abstract state representation, interpretable planning, and robust execution under partial observability. Evaluated in the AI2-THOR simulator, HOO-POMDP significantly outperforms existing reinforcement learning and handcrafted planning methods across diverse multi-object, multi-room rearrangement tasks—achieving higher task completion rates, stronger generalization to unseen object configurations and room layouts, and improved adaptability to environmental dynamics.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present an online planning framework for solving multi-object rearrangement problems in partially observable, multi-room environments. Current object rearrangement solutions, primarily based on Reinforcement Learning or hand-coded planning methods, often lack adaptability to diverse challenges. To address this limitation, we introduce a novel Hierarchical Object-Oriented Partially Observed Markov Decision Process (HOO-POMDP) planning approach. This approach comprises of (a) an object-oriented POMDP planner generating sub-goals, (b) a set of low-level policies for sub-goal achievement, and (c) an abstraction system converting the continuous low-level world into a representation suitable for abstract planning. We evaluate our system on varying numbers of objects, rooms, and problem types in AI2-THOR simulated environments with promising results.

Problem

• rearrangement
• multi-room environments
• partial observability

Innovation

• HOO-POMDP
• multi-room environments
• large-scale object rearrangement