See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View

2026-06-18

AI Summary

This work addresses the lack of effective evaluation for terminal approach capability in existing unmanned aerial vehicle (UAV) visual language navigation (VLN) after the target becomes visible. To this end, the authors propose the 3DG-VLN framework and introduce a novel UAV-VLN-FOV task formulation that decouples navigation into two stages: target discovery and terminal approach. The method integrates high-resolution forward- and downward-looking imagery, leveraging adaptive multi-view processing, dynamic 3D direction-guided waypoint prediction, and real-time orientation correction within a closed-loop system to achieve fine-grained visual localization and spatial alignment. Evaluated on a newly constructed high-resolution benchmark, the approach achieves a success rate of 13.82%, and real-world flight experiments demonstrate its effectiveness and practicality.

Abstract

UAV Vision-Language Navigation (UAV-VLN) is typically formulated as a holistic search-and-reach problem, where long-range target discovery and final target approach are optimized and evaluated jointly. This formulation makes it difficult to assess a critical capability of aerial embodied agents, namely whether a UAV can accurately ground a visible target and translate vision-language evidence into precise 3D motion once the target enters its field of view. To address this limitation, we introduce UAV-VLN-FOV, a target-visible navigation task that isolates the see-and-reach stage and enables a more diagnostic evaluation of terminal reaching ability. We further propose 3DG-VLN, a vision-language waypoint prediction framework guided by dynamic 3D direction cues to enhance fine-grained visual grounding and spatial direction alignment for precise target reaching. Specifically, 3DG-VLN adaptively processes high-resolution front-view and downward-view observations to preserve fine-grained visual and geometric details for target grounding. It also updates the target-relative direction online during closed-loop navigation, allowing the agent to maintain spatial alignment with the target and reduce accumulated direction drift. To support this task, we construct a dedicated high-resolution benchmark which contains 2,717 trajectories with target-oriented high-level instructions, high-resolution front-view and downward-view egocentric observations, and continuous 3D waypoint annotations. Experiments show that 3DG-VLN outperforms competitive UAV-VLN baselines, achieving a 13.82\% improvement in success rate. Real-world trials further demonstrate the potential of 3DG-VLN for practical see-and-reach navigation. The source code and benchmark are available at https://github.com/xuefanfu/3DG-VLN.

Problem

• UAV Vision-Language Navigation
• Field of View
• Target Grounding
• Precise Navigation
• 3D Motion

Innovation

• vision-language navigation
• 3D direction guidance
• fine-grained visual grounding
• UAV navigation
• field-of-view constrained task