Learning Approach to Efficient Vision-based Active Tracking of a Flying Target by an Unmanned Aerial Vehicle

2025-06-22

AI Summary

Addressing two key challenges in vision-based active UAV tracking—trade-offs between detection accuracy and real-time performance, and lack of environmental adaptability in maneuver decision-making—this paper proposes an integrated perception-decision framework. First, it combines a lightweight deep detection model with Kernelized Correlation Filters (KCF) to construct an efficient and robust target state estimation module. Second, it designs a reinforcement learning (RL) controller with environment-aware state representation (incorporating field-of-view constraints and motion consistency), a sparse action space, and a multi-objective reward function, enabling end-to-end neural control. Evaluations in AirSim simulations and on a physical lab platform demonstrate that, compared to a PID baseline, the proposed method increases average tracking duration by 42% and reduces mean tracking distance deviation by 58%, significantly enhancing robustness against highly maneuverable targets.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Autonomous tracking of flying aerial objects has important civilian and defense applications, ranging from search and rescue to counter-unmanned aerial systems (counter-UAS). Ground based tracking requires setting up infrastructure, could be range limited, and may not be feasible in remote areas, crowded cities or in dense vegetation areas. Vision based active tracking of aerial objects from another airborne vehicle, e.g., a chaser unmanned aerial vehicle (UAV), promises to fill this important gap, along with serving aerial coordination use cases. Vision-based active tracking by a UAV entails solving two coupled problems: 1) compute-efficient and accurate (target) object detection and target state estimation; and 2) maneuver decisions to ensure that the target remains in the field of view in the future time-steps and favorably positioned for continued detection. As a solution to the first problem, this paper presents a novel integration of standard deep learning based architectures with Kernelized Correlation Filter (KCF) to achieve compute-efficient object detection without compromising accuracy, unlike standalone learning or filtering approaches. The proposed perception framework is validated using a lab-scale setup. For the second problem, to obviate the linearity assumptions and background variations limiting effectiveness of the traditional controllers, we present the use of reinforcement learning to train a neuro-controller for fast computation of velocity maneuvers. New state space, action space and reward formulations are developed for this purpose, and training is performed in simulation using AirSim. The trained model is also tested in AirSim with respect to complex target maneuvers, and is found to outperform a baseline PID control in terms of tracking up-time and average distance maintained (from the target) during tracking.

Problem

• Efficient vision-based tracking of flying targets by UAVs
• Compute-efficient object detection and state estimation
• Reinforcement learning for UAV maneuver decisions

Innovation

• Integrates deep learning with Kernelized Correlation Filter
• Uses reinforcement learning for neuro-controller training
• Develops new state, action, reward formulations