SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigation

2026-01-11

arXiv.org

AI Summary

Zero-shot vision-and-language navigation (VLN) suffers from inefficient exploration and substantially lower performance compared to learning-based approaches due to its reliance on local observations without global spatial awareness. To address this limitation, this work proposes constructing a Spatial Scene Graph (SSG) prior to task execution, explicitly modeling the environment’s global structure and semantics. This is the first study to integrate SSG into zero-shot VLN, combining an agent-centric spatial map, compass-aligned visual representations, and a long-range object localization strategy to enable efficient zero-shot reasoning. Experimental results demonstrate that the proposed method significantly outperforms existing zero-shot approaches in both discrete and continuous environments, substantially narrowing the performance gap with state-of-the-art learning-based models.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Although learning-based vision-and-language navigation (VLN) agents can learn spatial knowledge implicitly from large-scale training data, zero-shot VLN agents lack this process, relying primarily on local observations for navigation, which leads to inefficient exploration and a significant performance gap. To deal with the problem, we consider a zero-shot VLN setting that agents are allowed to fully explore the environment before task execution. Then, we construct the Spatial Scene Graph (SSG) to explicitly capture global spatial structure and semantics in the explored environment. Based on the SSG, we introduce SpatialNav, a zero-shot VLN agent that integrates an agent-centric spatial map, a compass-aligned visual representation, and a remote object localization strategy for efficient navigation. Comprehensive experiments in both discrete and continuous environments demonstrate that SpatialNav significantly outperforms existing zero-shot agents and clearly narrows the gap with state-of-the-art learning-based methods. Such results highlight the importance of global spatial representations for generalizable navigation.

Problem

• zero-shot
• vision-and-language navigation
• spatial knowledge
• global spatial representation
• navigation efficiency

Innovation

• Spatial Scene Graph
• Zero-Shot Navigation
• Vision-and-Language Navigation
• Global Spatial Representation
• Remote Object Localization