From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation

2026-07-04

AI Summary

This study addresses the insufficiency of current vision-and-language navigation (VLN) evaluation protocols, which merely require agents to stop within 3 meters of a target while neglecting target visibility and final orientation, thereby undermining instance-level visual grounding. To bridge this “last-3-meter grounding gap,” the authors introduce REALM—a plug-and-play, architecture-agnostic refinement module that decouples long-horizon navigation from fine-grained target approach. REALM incorporates a visibility-aware stopping strategy and a region-to-entity alignment mechanism. Leveraging a newly curated REVERIE-AIM dataset comprising 180,000 short-range trajectories, the work proposes three instance-oriented evaluation metrics. Experiments demonstrate that REALM consistently enhances proximity accuracy and visual grounding success across four mainstream VLN backbones, confirming its generality and effectiveness.

Abstract

Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent's final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.

Problem

• Vision-and-Language Navigation
• instance-level grounding
• Last-3-Meter Grounding Gap
• target visibility
• proximity precision

Innovation

• instance-level grounding
• Last-3-Meter Grounding Gap
• REALM
• visibility-aware stopping
• vision-and-language navigation