Advancing DialNav through Automatic Embodied Dialog Augmentation

2026-06-18

AI Summary

This work addresses the limitations of existing dialogue-based navigation frameworks—such as DialNav—which suffer from severe data scarcity (only ~2K samples) and consequently struggle to achieve safe and efficient embodied navigation in real-world indoor environments. To overcome this, the authors introduce RAINbow, a large-scale, automatically constructed multi-turn dialogue navigation dataset comprising 238K samples, along with a dual-policy training mechanism and a localization model that integrates visual-language navigation (VLN) priors. This approach substantially enhances the agent’s performance within the dialogue-action closed loop, achieving state-of-the-art results with success rates of 58.24% (+89%) on Val Seen and 29.05% (+100%) on Val Unseen benchmarks.

Abstract

For embodied agents capable of physical interaction, the capability to create and understand dialog is crucial to ensure both safety and effectiveness. While DialNav~\cite{han2025dialnav} provides a framework for holistic evaluation of the dialog--execution loop in photorealistic indoor navigation, its performance remains limited by a critical scarcity of training data (2K episodes). To address this, we propose an automatic generation pipeline, and construct the \textbf{RAINbow} dataset, a large-scale training dataset with 238K episodes for DialNav. Our pipeline converts existing VLN datasets into multi-turn dialog and creates cost-efficient and high-quality dataset. Then, we introduce two additional complementary advances to unlock the data's full potential: (1) Dual-Strategy Training, a navigation training scheme to align the navigation training with the dynamic dialog-navigation loop, and (2) a localization model that leverages VLN knowledge. By combining these complementary solutions, our model substantially outperforms the baseline in success rate on both \textbf{Val Seen} (58.24, \textbf{+89\%}) and \textbf{Val Unseen} (29.05, \textbf{+100\%}) splits, establishing a new state of the art.

Problem

• embodied dialogue
• data scarcity
• indoor navigation
• dialog-execution loop
• training data

Innovation

• Automatic Dialog Augmentation
• RAINbow Dataset
• Dual-Strategy Training
• Embodied Dialog Navigation
• VLN-based Localization