YOPOv2-Tracker: An End-to-End Agile Tracking and Navigation Framework from Perception to Action

2025-05-11

AI Summary

Traditional quadrotor target tracking relies on a multi-stage pipeline—detection, mapping, navigation, and control—resulting in high latency and limited agility. This paper introduces the first interpretable end-to-end agile tracking framework that directly maps multimodal sensor inputs to safe, smooth control commands. Our approach explicitly embeds semantics of conventional modules by using motion primitives as anchors to jointly regress trajectory offsets and cost metrics; it backpropagates gradients directly through the trajectory optimization objective, eliminating reliance on expert demonstrations or reinforcement learning; and it integrates disturbance-compensation control with real-time embedded deployment. Evaluated in complex real-world environments—including forests and urban settings—the framework achieves highly robust, real-time tracking: end-to-end latency is reduced by 62%, response speed increases by 3.1×, and no offline annotation or simulation-based pretraining is required.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Tracking, profiling, and countermeasures against them

Abstract

Traditional target tracking pipelines including detection, mapping, navigation, and control are comprehensive but introduce high latency, limitting the agility of quadrotors. On the contrary, we follow the design principle of"less is more", striving to simplify the process while maintaining effectiveness. In this work, we propose an end-to-end agile tracking and navigation framework for quadrotors that directly maps the sensory observations to control commands. Importantly, leveraging the multimodal nature of navigation and detection tasks, our network maintains interpretability by explicitly integrating the independent modules of the traditional pipeline, rather than a crude action regression. In detail, we adopt a set of motion primitives as anchors to cover the searching space regarding the feasible region and potential target. Then we reformulate the trajectory optimization as regression of primitive offsets and associated costs considering the safety, smoothness, and other metrics. For tracking task, the trajectories are expected to approach the target and additional objectness scores are predicted. Subsequently, the predictions, after compensation for the estimated lumped disturbance, are transformed into thrust and attitude as control commands for swift response. During training, we seamlessly integrate traditional motion planning with deep learning by directly back-propagating the gradients of trajectory costs to the network, eliminating the need for expert demonstration in imitation learning and providing more direct guidance than reinforcement learning. Finally, we deploy the algorithm on a compact quadrotor and conduct real-world validations in both forest and building environments to demonstrate the efficiency of the proposed method.

Problem

• Reducing latency in quadrotor tracking and navigation systems
• Mapping sensory observations directly to control commands
• Maintaining interpretability while simplifying traditional tracking pipelines

Innovation

• End-to-end mapping of sensory observations to control commands
• Motion primitives as anchors for trajectory optimization
• Integration of motion planning with deep learning gradients