Decentralized End-to-End Multi-AAV Pursuit Using Predictive Spatio-Temporal Observation via Deep Reinforcement Learning

2026-03-25

AI Summary

This work addresses the challenge of decentralized cooperative pursuit by multiple autonomous aerial vehicles (AAVs) in cluttered environments using only local, noisy sensing. The authors propose an end-to-end multi-agent reinforcement learning framework that directly maps raw LiDAR data to continuous control commands. The key innovation lies in the predictive spatiotemporal observation (PSTO) representation, which unifies the encoding of obstacles, target intent, and teammate motion, thereby integrating perception and decision-making into a single pipeline. Notably, the approach operates without access to privileged information and enables zero-shot policy transfer across different team sizes. Simulations demonstrate superior capture efficiency and success rates compared to existing methods that rely on privileged information. Furthermore, real-world outdoor experiments with a quadrotor swarm validate the system’s fully autonomous, onboard-only capability for cooperative pursuit.

Technology Category

• Multiagent Systems: Teamwork
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Decentralized cooperative pursuit in cluttered environments is challenging for autonomous aerial swarms, especially under partial and noisy perception. Existing methods often rely on abstracted geometric features or privileged ground-truth states, and therefore sidestep perceptual uncertainty in real-world settings. We propose a decentralized end-to-end multi-agent reinforcement learning (MARL) framework that maps raw LiDAR observations directly to continuous control commands. Central to the framework is the Predictive Spatio-Temporal Observation (PSTO), an egocentric grid representation that aligns obstacle geometry with predictive adversarial intent and teammate motion in a unified, fixed-resolution projection. Built on PSTO, a single decentralized policy enables agents to navigate static obstacles, intercept dynamic targets, and maintain cooperative encirclement. Simulations demonstrate that the proposed method achieves superior capture efficiency and competitive success rates compared to state-of-the-art learning-based approaches relying on privileged obstacle information. Furthermore, the unified policy scales seamlessly across different team sizes without retraining. Finally, fully autonomous outdoor experiments validate the framework on a quadrotor swarm relying on only onboard sensing and computing.

Problem

• decentralized pursuit
• perceptual uncertainty
• autonomous aerial swarms
• cluttered environments
• multi-agent cooperation

Innovation

• Predictive Spatio-Temporal Observation
• Decentralized MARL
• End-to-End LiDAR Control
• Autonomous Aerial Swarm
• Cooperative Pursuit