AgilePE: Autonomous UAV Pursuit-Evasion via Self-Play Reinforcement Learning

2026-08-14

AI Summary

This study addresses the challenges of high-dimensional dynamic interactions and agile decision-making in autonomous UAV pursuit-evasion by proposing an end-to-end control framework based on self-play reinforcement learning. By integrating Prioritized Fictitious Self-Play (PFSP), a CTBR control architecture, and hardware-aligned simulation, the method enables direct mapping from observations to control commands. This approach effectively mitigates the sim-to-real gap, achieving zero-shot deployment on physical platforms. Experimental results demonstrate that the system successfully emergent complex tactics, including evasion and encirclement, significantly enhancing UAV autonomous maneuverability and real-world transfer performance in dynamic adversarial environments.

Abstract

Autonomous pursuit-evasion is a fundamental challenge for Unmanned Aerial Vehicles (UAVs), requiring rapid decision-making under tightly coupled dynamics and continuously changing opponent behaviors. Traditional rule-based or differential-game approaches often struggle with high-dimensional aerial interactions and agile maneuvering. We present AgilePE, a complete system for autonomous UAV pursuit-evasion via self-play reinforcement learning. AgilePE integrates agile low-level control, competitive policy optimization, and sim-to-real deployment in a unified framework. The policy directly maps onboard state observations to Collective Thrust and Body Rates (CTBR) commands, enabling end-to-end agile maneuvering without intermediate trajectory planners or waypoint controllers. For training, we use competitive self-play with Prioritized Fictitious Self-Play (PFSP) and a diversified opponent pool, enabling agents to improve against historical policies while stabilizing optimization and reducing policy oscillation. This process leads to the emergence of sophisticated pursuit and evasion strategies. For real-world deployment, we develop a hardware-aligned simulation pipeline that models actuator-response dynamics, communication latency, and domain randomization. The learned policies transfer zero-shot to real quadrotors without task-specific tuning. Real-world experiments reproduce pursuit-evasion tactics observed in simulation, including rapid dodging and flanking, and demonstrate interactive two-agent zero-shot deployment.

Problem

• UAV pursuit-evasion
• autonomous decision-making
• agile maneuvering
• high-dimensional aerial interactions

Innovation

• Self-Play Reinforcement Learning
• End-to-End Control
• Sim-to-Real Transfer
• Prioritized Fictitious Self-Play
• UAV Pursuit-Evasion