DecoVLN: Decoupling Observation, Reasoning, and Correction for Vision-and-Language Navigation

2026-03-13

AI Summary

This work addresses the challenges of long-term memory construction and error accumulation in vision-and-language navigation by proposing an adaptive memory refinement mechanism. The approach employs a unified scoring function to select salient historical frames and integrates geodesic distance-based filtering to discard low-quality state-action pairs for corrective fine-tuning. By effectively decoupling perception, reasoning, and error correction into distinct yet coordinated processes, the method significantly enhances navigation robustness and long-horizon stability. Experimental results demonstrate substantial improvements in both navigation success rates and error-correction efficiency across simulated and real-world environments.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: Language and Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Vision-and-Language Navigation (VLN) requires agents to follow long-horizon instructions and navigate complex 3D environments. However, existing approaches face two major challenges: constructing an effective long-term memory bank and overcoming the compounding errors problem. To address these issues, we propose DecoVLN, an effective framework designed for robust streaming perception and closed-loop control in long-horizon navigation. First, we formulate long-term memory construction as an optimization problem and introduce adaptive refinement mechanism that selects frames from a historical candidate pool by iteratively optimizing a unified scoring function. This function jointly balances three key criteria: semantic relevance to the instruction, visual diversity from the selected memory, and temporal coverage of the historical trajectory. Second, to alleviate compounding errors, we introduce a state-action pair-level corrective finetuning strategy. By leveraging geodesic distance between states to precisely quantify deviation from the expert trajectory, the agent collects high-quality state-action pairs in the trusted region while filtering out the polluted data with low relevance. This improves both the efficiency and stability of error correction. Extensive experiments demonstrate the effectiveness of DecoVLN, and we have deployed it in real-world environments.

Problem

• Vision-and-Language Navigation
• long-term memory
• compounding errors
• 3D navigation
• instruction following

Innovation

• long-term memory optimization
• adaptive refinement
• compounding error correction
• state-action corrective finetuning
• vision-and-language navigation