$NavA^3$: Understanding Any Instruction, Navigating Anywhere, Finding Anything

2025-08-06

AI Summary

Existing embodied navigation systems are largely confined to predefined targets or simple commands, failing to address open-ended, complex, and long-horizon high-level semantic navigation demands prevalent in real-world environments. Method: We propose the first open-vocabulary, spatially aware, and cross-form-factor deployable long-horizon embodied navigation framework. It integrates a reasoning-capable vision-language model (Reasoning-VLM) with a spatially grounded object affordance model (NaviAfford), implementing a global–local two-stage policy: the global stage parses high-level instructions and fuses 3D scene understanding, while the local stage performs fine-grained object localization and affordance-aware reasoning. Contribution/Results: Trained on a million-scale real-scene dataset, our framework achieves state-of-the-art performance across multiple robotic platforms. It is the first to enable end-to-end, instruction-driven, cross-form-factor, long-horizon general embodied navigation—demonstrating the feasibility of universal navigation in open environments.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Visual Reasoning & Symbolic Representations
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Embodied navigation is a fundamental capability of embodied intelligence, enabling robots to move and interact within physical environments. However, existing navigation tasks primarily focus on predefined object navigation or instruction following, which significantly differs from human needs in real-world scenarios involving complex, open-ended scenes. To bridge this gap, we introduce a challenging long-horizon navigation task that requires understanding high-level human instructions and performing spatial-aware object navigation in real-world environments. Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary. In this paper, we propose $NavA^3$, a hierarchical framework divided into two stages: global and local policies. In the global policy, we leverage the reasoning capabilities of Reasoning-VLM to parse high-level human instructions and integrate them with global 3D scene views. This allows us to reason and navigate to regions most likely to contain the goal object. In the local policy, we have collected a dataset of 1.0 million samples of spatial-aware object affordances to train the NaviAfford model (PointingVLM), which provides robust open-vocabulary object localization and spatial awareness for precise goal identification and navigation in complex environments. Extensive experiments demonstrate that $NavA^3$ achieves SOTA results in navigation performance and can successfully complete longhorizon navigation tasks across different robot embodiments in real-world settings, paving the way for universal embodied navigation. The dataset and code will be made available. Project website: https://NavigationA3.github.io/.

Problem

• Bridging gap in complex real-world navigation tasks
• Enhancing high-level human instruction comprehension
• Improving open-vocabulary object localization accuracy

Innovation

• Hierarchical framework with global and local policies
• Uses Reasoning-VLM for high-level instruction comprehension
• Trains NaviAfford model for open-vocabulary object localization