LLM Meets the Sky: Heuristic Multi-Agent Reinforcement Learning for Secure Heterogeneous UAV Networks

2025-07-23

AI Summary

To address physical-layer security in energy-constrained heterogeneous unmanned aerial vehicle networks (HetUAVNs), this paper proposes a hierarchical optimization framework that maximizes the secrecy rate while guaranteeing communication confidentiality. At the inner level, a joint semi-definite relaxation (SDR) and difference-of-convex (DC) programming approach computes the optimal secrecy precoding for fixed UAV positions. At the outer level, a large language model (LLM)-guided heuristic multi-agent reinforcement learning method generates lightweight expert policies, enabling energy-efficient, collaborative trajectory optimization without real-time LLM invocation. The framework jointly optimizes heterogeneous resource allocation, security performance, and energy efficiency. Extensive experiments demonstrate that the proposed method consistently outperforms state-of-the-art baselines across diverse network scales and random seeds, achieving a 23.6% improvement in secrecy rate and a 31.4% gain in secrecy rate per unit energy consumption, thereby exhibiting strong robustness and practical applicability.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This work tackles the physical layer security (PLS) problem of maximizing the secrecy rate in heterogeneous UAV networks (HetUAVNs) under propulsion energy constraints. Unlike prior studies that assume uniform UAV capabilities or overlook energy-security trade-offs, we consider a realistic scenario where UAVs with diverse payloads and computation resources collaborate to serve ground terminals in the presence of eavesdroppers. To manage the complex coupling between UAV motion and communication, we propose a hierarchical optimization framework. The inner layer uses a semidefinite relaxation (SDR)-based S2DC algorithm combining penalty functions and difference-of-convex (d.c.) programming to solve the secrecy precoding problem with fixed UAV positions. The outer layer introduces a Large Language Model (LLM)-guided heuristic multi-agent reinforcement learning approach (LLM-HeMARL) for trajectory optimization. LLM-HeMARL efficiently incorporates expert heuristics policy generated by the LLM, enabling UAVs to learn energy-aware, security-driven trajectories without the inference overhead of real-time LLM calls. The simulation results show that our method outperforms existing baselines in secrecy rate and energy efficiency, with consistent robustness across varying UAV swarm sizes and random seeds.

Problem

• Maximize secrecy rate in heterogeneous UAV networks
• Address energy-security trade-offs in UAV collaboration
• Optimize UAV trajectories with LLM-guided reinforcement learning

Innovation

• Hierarchical optimization framework for UAV networks
• SDR-based S2DC algorithm for secrecy precoding
• LLM-guided heuristic MARL for trajectory optimization