User Association and Coordinated Beamforming in Cognitive Aerial-Terrestrial Networks: A Safe Reinforcement Learning Approach

2025-02-19

AI Summary

In cognitive air–ground–space integrated networks, aerial users suffer severe interference from terrestrial base stations (BSs), necessitating strict interference temperature constraints to protect them. Method: We jointly optimize terrestrial user association and multi-BS covariance-based coordinated beamforming to maximize the sum rate of terrestrial users while guaranteeing hard interference constraints at aerial users. To address the challenges of partial observability and safety-critical constraints in spectrum sharing, we formulate the problem as a networked constrained partially observable Markov game and propose a parameter-free safe deep reinforcement learning framework—achieving rigorous constraint satisfaction in a single training phase without iterative hyperparameter tuning of penalty weights. Contribution/Results: Compared with conventional two-stage optimization, our approach achieves significant gains in terrestrial sum rate, maintains aerial users’ average received interference power consistently below the threshold, and eliminates training overhead entirely—reducing deployment cost by 100%.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Cognitive aerial-terrestrial networks (CATNs) offer a solution to spectrum scarcity by sharing spectrum between aerial and terrestrial networks. However, aerial users (AUs) experience significant interference from numerous terrestrial base stations (BSs). To alleviate such interference, we investigate a user association and coordinated beamforming (CBF) problem in CATN, where the aerial network serves as the primary network sharing its spectrum with the terrestrial network. Specifically, we maximize the sum rate of the secondary terrestrial users (TUs) under the interference temperature constraints of the AUs. Traditional iterative optimization schemes are impractical due to their high computational complexity and information exchange overhead. Although deep reinforcement learning (DRL) based schemes can address these challenges, their performance is sensitive to the weights of the weighted penalty terms for violating constraints in the reward function. Motivated by these issues, we propose a safe DRL-based user association and CBF scheme for CATN, eliminating the need for training multiple times to find the optimal penalty weight before actual deployment. Specifically, the CATN is modeled as a networked constrained partially observable Markov game. Each TU acts as an agent to choose its associated BS, and each BS acts as an agent to decide its beamforming vectors, aiming to maximize the reward while satisfying the safety constraints introduced by the interference constraints of the AUs. By exploiting a safe DRL algorithm, the proposed scheme incurs lower deployment expenses than the penalty-based DRL schemes since only one training is required before actual deployment. Simulation results show that the proposed scheme can achieve a higher sum rate of TUs than a two-stage optimization scheme while the average received interference power of the AUs is generally below the threshold.

Problem

• Optimizes user association and beamforming in CATN
• Maximizes terrestrial users' sum rate under interference constraints
• Proposes safe DRL for reduced computational complexity

Innovation

• Safe DRL algorithm
• Coordinated beamforming optimization
• Single training deployment