From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation

2026-08-10

AI Summary

This work addresses key challenges in vision-and-language navigation for unmanned aerial vehicles, including weak semantic grounding of instruction-relevant landmarks, insufficient exploitation of long-horizon history, and decision instability caused by local traps. To tackle these issues, the authors propose a unified semantic-to-decision framework comprising an instruction-anchored semantic enhancement module, a relevance-aware dynamic history aggregation strategy, and a topology-aware decision mechanism integrating multidimensional rewards. The framework effectively fuses object-level semantics, relative spatial encoding, and structured landmark cues, while enhancing robustness through population-based relative policy optimization. Evaluated on the AerialVLN and OpenFly benchmarks, the method achieves state-of-the-art performance, significantly improving both navigation success rate and path fidelity to instructions.

Abstract

UAV vision-language navigation (UAV-VLN) focuses on enabling an aerial agent to follow natural-language instructions in open 3D environments from egocentric visual observations. Current approaches suffer from three coupled issues: weak grounding of instruction-relevant landmarks in visual observations, insufficient exploitation of long-horizon history, and unstable decisions under local traps or repeated exploration. To address these issues, we propose a unified semantic-to-decision framework. First, we present an instruction-grounded semantic enhancement module that injects object-level semantics and relative spatial cues into the current observation state. Subsequently, we develop a relevance-aware dynamic temporal aggregation strategy that reweights the full history buffer while converting a few high-relevance frames into structured landmark prompts for the decoder. Finally, we devise a topology-aware decision method that combines local-optimum cognition with group-relative policy optimization under progress, goal, semantic, and path-compliance rewards. Experiments on the widely used AerialVLN and OpenFly benchmarks clearly demonstrate that our method achieves state-of-the-art performance.

Problem

• UAV vision-language navigation
• semantic grounding
• long-horizon history
• decision stability
• local traps

Innovation

• semantic grounding
• long-horizon navigation
• dynamic temporal aggregation
• topology-aware decision
• vision-language navigation