Deep Reinforcement Learning based Autonomous Decision-Making for Cooperative UAVs: A Search and Rescue Real World Application

2025-02-27

AI Summary

To address the challenge of collaborative search-and-rescue (SAR) by multiple UAVs in GNSS-denied indoor environments, this paper proposes an end-to-end framework integrating autonomous navigation with dynamic task allocation. Methodologically: (1) we design an artificial potential field (APF)-enhanced Twin Delayed Deep Deterministic Policy Gradient (TD3) navigation policy, incorporating APF-based rewards for robust obstacle avoidance and precise corridor localization; (2) we develop a distributed task allocation mechanism based on Graph Convolutional Networks (GCNs), enabling real-time response to dynamic targets; (3) we adopt a tightly coupled multi-sensor localization scheme fusing LiDAR-SLAM and depth-camera data, significantly improving pose estimation accuracy in structured indoor settings. Evaluated in the NATO Sapience 2024 international competition, the system achieved first place globally, demonstrating superior robustness, adaptability, and coordination performance in complex indoor SAR scenarios.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper proposes a holistic framework for autonomous guidance, navigation, and task distribution among multi-drone systems operating in Global Navigation Satellite System (GNSS)-denied indoor settings. We advocate for a Deep Reinforcement Learning (DRL)-based guidance mechanism, utilising the Twin Delayed Deep Deterministic Policy Gradient algorithm. To improve the efficiency of the training process, we incorporate an Artificial Potential Field (APF)-based reward structure, enabling the agent to refine its movements, thereby promoting smoother paths and enhanced obstacle avoidance in indoor contexts. Furthermore, we tackle the issue of task distribution among cooperative UAVs through a DRL-trained Graph Convolutional Network (GCN). This GCN represents the interactions between drones and tasks, facilitating dynamic and real-time task allocation that reflects the current environmental conditions and the capabilities of the drones. Such an approach fosters effective coordination and collaboration among multiple drones during search and rescue operations or other exploratory endeavours. Lastly, to ensure precise odometry in environments lacking GNSS, we employ Light Detection And Ranging Simultaneous Localisation and Mapping complemented by a depth camera to mitigate the hallway problem. This integration offers robust localisation and mapping functionalities, thereby enhancing the systems dependability in indoor navigation. The proposed multi-drone framework not only elevates individual navigation capabilities but also optimises coordinated task allocation in complex, obstacle-laden environments. Experimental evaluations conducted in a setup tailored to meet the requirements of the NATO Sapience Autonomous Cooperative Drone Competition demonstrate the efficacy of the proposed system, yielding outstanding results and culminating in a first-place finish in the 2024 Sapience competition.

Problem

• Autonomous decision-making for multi-drone systems
• Task distribution among cooperative UAVs
• Precise odometry in GNSS-denied indoor environments

Innovation

• Deep Reinforcement Learning guidance
• Graph Convolutional Network task allocation
• LIDAR with depth camera localization