Right Place, Right Time! Generalizing ObjectNav to Dynamic Environments with Portable Targets

2024-03-14

AI Summary

Real-world objects are often mobile, yet existing embodied navigation tasks—such as ObjectNav—assume static targets, limiting robustness in dynamic environments. To address this, we propose P-ObjectNav, the first paradigm extending ObjectNav to scenarios with *mobile* targets. Our method introduces a dynamic topological scene graph that explicitly models multi-modal object motion patterns. We further establish DynObjectNav—the first embodied AI navigation benchmark for mobile targets—accompanied by dedicated evaluation metrics. The framework integrates Matterport3D-based dynamic scene generation, heuristic search, reinforcement learning, and large language model–guided navigation. Experiments demonstrate significant improvements in navigation robustness under object mobility. All code and datasets are publicly released to advance research in dynamic embodied intelligence.

Technology Category

• Intelligent Robots: Embodied AI
• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

ObjectNav is a popular task in Embodied AI, where an agent navigates to a target object in an unseen environment. Prior literature makes the assumption of a static environment with stationary objects, which lacks realism. To address this, we present a novel formulation to generalize ObjectNav to dynamic environments with non-stationary objects, and refer to it as Portable ObjectNav or P-ObjectNav. In our formulation, we first address several challenging issues with dynamizing existing topological scene graphs by developing a novel method that introduces multiple transition behaviors to portable objects in the scene. We use this technique to dynamize Matterport3D, a popular simulator for evaluating embodied tasks. We then present a benchmark for P-ObjectNav using a combination of heuristic, reinforcement learning, and Large Language Model (LLM)-based navigation approaches on the dynamized environment, while introducing novel evaluation metrics tailored for our task. Our work fundamentally challenges the"static-environment"notion of prior ObjectNav work; the code and dataset for P-ObjectNav will be made publicly available to foster research on embodied navigation in dynamic scenes. We provide an anonymized repository for our code and dataset: https://anonymous.4open.science/r/PObjectNav-1C6D.

Problem

• Addressing navigation in dynamic environments with moving objects
• Introducing Object Transition Graphs to dynamize static topological graphs
• Evaluating agent adaptability and decision-making strategies in dynamic scenes

Innovation

• Introduces Object Transition Graphs (OTGs)
• Simulates portable targets with structured routes
• Evaluates navigation using RL and LLM approaches