A Cross-Scene Benchmark for Open-World Drone Active Tracking

2024-12-01

arXiv.org

AI Summary

To address the lack of a unified benchmark and insufficient robustness in active visual target tracking by UAVs within open-world dynamic environments, this paper introduces DAT—the first cross-scene, cross-domain open-world benchmark for UAV-based active tracking—comprising 24 visually complex environments and high-fidelity dynamical modeling. We propose R-VAT, a reinforcement learning (RL)-based method featuring a novel curriculum learning strategy and a target-centered reward function. Our key contributions are: (1) the first standardized benchmark enabling systematic generalization evaluation; and (2) an RL framework specifically designed to enhance long-term tracking stability under strong environmental disturbances. Experimental results demonstrate that R-VAT achieves approximately 400% higher cumulative reward than state-of-the-art methods, significantly improving tracking robustness and generalization capability in open-world settings.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations, providing a more practical solution for effective tracking in dynamic environments. However, accurate Drone Visual Active Tracking using reinforcement learning remains challenging due to the absence of a unified benchmark, the complexity of open-world environments with frequent interference, and the diverse motion behavior of dynamic targets. To address these issues, we propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT. The DAT benchmark provides 24 visually complex environments to assess the algorithms' cross-scene and cross-domain generalization abilities, and high-fidelity modeling of realistic robot dynamics. Additionally, we propose a reinforcement learning-based drone tracking method called R-VAT, which aims to improve the performance of drone tracking targets in complex scenarios. Specifically, inspired by curriculum learning, we introduce a Curriculum-Based Training strategy that progressively enhances the agent tracking performance in vast environments with complex interference. We design a goal-centered reward function to provide precise feedback to the drone agent, preventing targets farther from the center of view from receiving higher rewards than closer ones. This allows the drone to adapt to the diverse motion behavior of open-world targets. Experiments demonstrate that the R-VAT has about 400% improvement over the SOTA method in terms of the cumulative reward metric.

Problem

• Addresses drone visual tracking challenges in open-world dynamic environments
• Develops benchmark for realistic air-to-ground tracking with human-like behaviors
• Proposes goal-centered reinforcement learning to improve tracking success rates

Innovation

• Proposed DAT benchmark for open-world drone tracking
• Designed Goal-Centered Reward for viewpoint feedback
• Introduced Curriculum-Based Training for progressive learning