RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation

2026-08-10

AI Summary

This work addresses key challenges in vision-language navigation for unmanned aerial vehicles, where end-to-end policies suffer from insufficient corrective supervision during closed-loop execution, and reinforcement learning (RL) struggles with low sample efficiency, poor generalization to long-tail scenarios, and policy distribution shift. To overcome these limitations, the authors propose a failure-aware RL post-training framework that optimizes grammar-constrained autoregressive action outputs at the token level, incorporates a failure-case replay mechanism to enhance error correction, and employs a two-stage long-tail curriculum with reference policy regularization to improve generalization. This approach uniquely integrates failure awareness with token-level RL, achieving state-of-the-art performance on the TravelUAV benchmark—using only ~30% of the training interaction data—and yielding absolute success rate gains of 3.12–8.37 percentage points across both seen and unseen map and object configurations.

Abstract

Unmanned aerial vehicle vision-language navigation (UAV-VLN) requires agents to translate visual observations and language instructions into reliable flight actions in complex environments. Although recent end-to-end UAV vision-language-action (UAV-VLA) policies reduce reliance on separately designed perception, planning, and control modules, their behavior-cloning objectives provide limited corrective supervision for interactive closed-loop execution. Reinforcement learning (RL) offers a promising solution, while its effectiveness is constrained by inefficient use of samples, long-tailed scene distributions, and policy distribution shift during optimization. To this end, we propose RecoverFly, a failure-aware RL post-training framework for end-to-end UAV-VLA policies. Specifically, RecoverFly adapts token-level RL for stable optimization of grammar-constrained autoregressive UAV actions, revisits unresolved failure cases to strengthen corrective learning and sample utilization, and combines a two-stage long-tail scene curriculum with reference-policy regularization to improve scene adaptation while preserving acquired capabilities. Experiments on the TravelUAV benchmark demonstrate that RecoverFly achieves the best performance on the seen, unseen-map, and unseen-object splits. Moreover, compared to the AerialVLA initialization, RecoverFly improves success rate by 3.12 to 8.37 percentage points under a total rollout budget of about 30\% of the training-set size, validating its effectiveness, robustness, and generalization capabilities.

Problem

• UAV-VLN
• reinforcement learning
• failure recovery
• distribution shift
• long-tailed scenes

Innovation

• failure-aware reinforcement learning
• token-level RL
• long-tail scene curriculum
• reference-policy regularization
• UAV vision-language navigation