DaViNCi: A Dataset Towards Outdoor Vision-and-Language Navigation with Continuous Actions and Dynamic Elements

2026-08-12

AI Summary

This work addresses the limitations of existing outdoor Vision-and-Language Navigation (VLN) datasets, which rely on static, discrete topological graphs and fail to capture the dynamics of real-world environments, thereby hindering sim-to-real transfer. To bridge this gap, we introduce DaViNCi—the first outdoor VLN dataset that supports continuous action spaces and interaction with dynamic obstacles. Built upon a high-fidelity simulator, DaViNCi comprises six photorealistic maps and 6,933 navigation trajectories, uniquely integrating continuous control with dynamic environmental elements in an outdoor VLN setting. Experimental results demonstrate that agent performance on DaViNCi exhibits a success rate drop of over 10% compared to prior datasets, with an even more pronounced decline under continuous-action settings, underscoring the dataset’s realism, challenge, and practical relevance for advancing robust outdoor navigation systems.

Abstract

Vision-and-Language Navigation (VLN) has progressively expanded from indoor to outdoor environments. However, existing outdoor VLN datasets still rely on fixed discrete topological graphs for construction. It fails to align with the rapidly changing real-world outdoor environments and impedes the sim-to-real transfer of VLN agents. To address this limitation, we propose DaViNCi (\textbf{D}yn\textbf{a}mic \textbf{Vi}sion-and-Language \textbf{N}avigation in \textbf{C}ont\textbf{i}nuous Environment), the first outdoor VLN dataset that simultaneously introduces both continuous and dynamic factors. The agent not only moves in the outdoor environment using continuous actions but is also required to handle unpredictable dynamic elements. The dataset encompasses six distinct maps with a total of 6,933 trajectories. Through comprehensive comparative experiments, we find that the success rate on DaViNCi decreased by more than 10\% in discrete environments compared to previous datasets. And there is an even greater decline in continuous settings, demonstrating the challenge of DaViNCi. Furthermore, we clarify the impact of action granularity and dynamic elements. These results demonstrate the practical value of DaViNCi in advancing outdoor VLN toward more realistic environments. The website is https://xzh0312.github.io/DaViNCi/.

Problem

• Vision-and-Language Navigation
• outdoor environments
• continuous actions
• dynamic elements
• sim-to-real transfer

Innovation

• continuous action space
• dynamic environments
• outdoor vision-and-language navigation
• sim-to-real transfer
• navigation dataset