A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation

2026-07-09

AI Summary

This work addresses the limited robustness of existing Vision-and-Language Navigation (VLN) methods in open real-world environments and the absence of a systematic taxonomy with empirical validation. The authors propose the first orthogonal classification framework for VLN, structured along two axes: action paradigms (hierarchical vs. monolithic) and model paradigms (discriminative vs. generative). Building upon this framework, they conduct a comprehensive review and large-scale evaluation on real robotic platforms across ten diverse real-world scenes. Experimental results reveal a significant sim-to-real performance gap: while monolithic RGB-based methods achieve a 61% success rate in simulation, their real-world performance drops sharply to 22%. In contrast, hierarchical approaches attain a 51% success rate in real environments, substantially outperforming monolithic counterparts and demonstrating superior robustness in practical deployment.

Abstract

Navigation is a fundamental capability of autonomous systems, yet most existing approaches rely on highly structured models and strong prior assumptions, limiting their robustness in open and uncertain real-world environments. Vision-and-Language Navigation (VLN) offers a promising direction by enabling robots to integrate natural language understanding with visual perception in a data-driven manner. Although VLN has attracted increasing research attention, systematic methodological taxonomy and real-world validation remain limited. This survey presents a comprehensive review of VLN research. Specifically, state-of-the-art methods are organized along two orthogonal dimensions: action paradigms, including hierarchical and monolithic frameworks, and model paradigms, including discriminative and generative approaches. A critical analysis of their respective strengths and limitations is provided. Additionally, we conduct a systematic real-world evaluation of representative VLN system configurations on a physical robotic platform. Experiments across ten diverse real-world scenes show a substantial performance gap between simulation and real-world deployment under the tested configurations: a representative monolithic RGB-only method achieves 61% success in simulation but drops to 22% in real-world deployment, while a hierarchical framework achieves a higher real-world success rate of 51%, suggesting stronger robustness in our evaluation setting. Finally, we highlight key challenges in perception, decision-making, and control that must be addressed in future research.

Problem

• Vision-and-Language Navigation
• real-world evaluation
• embodied AI
• simulation-to-reality gap
• autonomous navigation

Innovation

• Vision-and-Language Navigation
• real-world evaluation
• hierarchical framework
• embodied AI
• simulation-to-reality gap