NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularization

2025-07-14

AI Summary

Language-guided navigation research is hindered by the scarcity of authentic human instructions and the low quality of synthetic instruction data. To address this, we propose a modular action–scene–object framework that enables high-fidelity natural language instruction generation for complex navigation paths. Furthermore, we design NavInstrCritic—a fully automated, annotation-free multi-dimensional evaluation system—that jointly models instruction–trajectory alignment, semantic consistency, and linguistic diversity. Crucially, NavInstrCritic decouples instruction generation from navigation policy learning, thereby enhancing instruction richness, accuracy, and cross-domain generalization. Experimental results demonstrate that our generated instructions significantly outperform existing synthetic baselines across multiple metrics. Moreover, NavInstrCritic achieves higher correlation with human judgments than prior automatic evaluators. Our framework provides a scalable, reproducible infrastructure for both instruction data synthesis and evaluation, advancing large-scale embodied intelligence research.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Language-guided navigation is a cornerstone of embodied AI, enabling agents to interpret language instructions and navigate complex environments. However, expert-provided instructions are limited in quantity, while synthesized annotations often lack quality, making them insufficient for large-scale research. To address this, we propose NavComposer, a novel framework for automatically generating high-quality navigation instructions. NavComposer explicitly decomposes semantic entities such as actions, scenes, and objects, and recomposes them into natural language instructions. Its modular architecture allows flexible integration of state-of-the-art techniques, while the explicit use of semantic entities enhances both the richness and accuracy of instructions. Moreover, it operates in a data-agnostic manner, supporting adaptation to diverse navigation trajectories without domain-specific training. Complementing NavComposer, we introduce NavInstrCritic, a comprehensive annotation-free evaluation system that assesses navigation instructions on three dimensions: contrastive matching, semantic consistency, and linguistic diversity. NavInstrCritic provides a holistic evaluation of instruction quality, addressing limitations of traditional metrics that rely heavily on expert annotations. By decoupling instruction generation and evaluation from specific navigation agents, our method enables more scalable and generalizable research. Extensive experiments provide direct and practical evidence for the effectiveness of our method.

Problem

• Generating high-quality navigation instructions automatically
• Enhancing instruction richness via action-scene-object modularization
• Evaluating instructions without relying on expert annotations

Innovation

• Modularizes actions, scenes, and objects
• Generates instructions without domain-specific training
• Evaluates instructions without expert annotations