Game-Theoretic Multi-Agent Reinforcement Learning for Swarm Trajectory Planning in Low-Altitude Wireless Networks

2026-06-15

AI Summary

This work addresses the strong trajectory coupling and congestion arising from massive UAV swarms sharing 5G NR resource blocks in multi-cell low-altitude wireless networks. It formulates cooperative trajectory planning for the first time as a cooperative stochastic congestion game, introducing a joint utility function that integrates communication throughput and task completion efficiency. Leveraging the centralized training with decentralized execution (CTDE) framework, the solution is obtained via the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm. Experimental results demonstrate that the proposed approach significantly outperforms QMIX, independent Q-learning, and random policies in terms of aggregate utility and task success rate, achieving stable convergence within a limited training budget and effectively enabling joint optimization of communication and mission objectives.

Abstract

The Low-Altitude Economy (LAE) is rapidly expanding, giving rise to low-altitude wireless networks (LAWNs), where large-scale cellular-connected unmanned aerial vehicle (UAV) deployments support heterogeneous mission-critical applications over multi-cell ground base station (GBS) infrastructures. To ensure mission success, each UAV must jointly optimize communication throughput and mission completion efficiency. In fifth-generation (5G) new radio (NR) systems, the equal resource block (RB) allocation policy induces strong strategic coupling among UAV trajectories: when a UAV enters a GBS cell, it reduces the RB share available to all co-served UAVs, thereby altering their achievable rates and trajectory incentives through shared wireless resources. Existing studies either ignore this coupling or focus on single-cell infrastructure, leaving the multi-cell, congestion-aware UAV trajectory planning problem insufficiently addressed. To fill this gap, we formulate the problem as a cooperative stochastic congestion game with a communication-and-mission-aware utility function, and propose a centralized-training decentralized-execution multi-agent proximal policy optimization (CTDE-MAPPO) algorithm to maximize social welfare under multi-cell RB congestion. Simulation results show that the proposed method outperforms QMIX, independent Q-learning, and random baselines in terms of aggregate utility and mission success rate, while achieving stable convergence within practical training budgets.

Problem

• multi-cell UAV trajectory planning
• resource block congestion
• strategic coupling
• low-altitude wireless networks
• mission-critical applications

Innovation

• multi-agent reinforcement learning
• congestion game
• swarm trajectory planning
• low-altitude wireless networks
• CTDE-MAPPO