Intelligent Spectrum Sharing in Integrated TN-NTNs: A Hierarchical Deep Reinforcement Learning Approach

2025-03-09

AI Summary

To address the multi-layer heterogeneous spectrum sharing challenge in integrated space-air-ground networks, this paper proposes a hierarchical deep reinforcement learning (HDRL) framework for high-dimensional, dynamic spectrum allocation and interference management under coordinated terrestrial networks (TNs) and non-terrestrial networks (NTNs). The method introduces a cross-layer policy decoupling mechanism enabling joint optimization across multiple time scales, thereby overcoming the scalability limitations of conventional single-layer DRL in complex dynamic environments. It integrates multi-agent cooperative decision-making, dynamic spectrum sensing, and abstracted heterogeneous network modeling. Simulation results demonstrate a 37% improvement in spectrum efficiency and a 52% reduction in interference, significantly outperforming traditional optimization and single-layer DRL approaches. These results validate the robustness and scalability of the proposed framework in dynamic multi-layer networking scenarios.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Integrating non-terrestrial networks (NTNs) with terrestrial networks (TNs) is key to enhancing coverage, capacity, and reliability in future wireless communications. However, the multi-tier, heterogeneous architecture of these integrated TN-NTNs introduces complex challenges in spectrum sharing and interference management. Conventional optimization approaches struggle to handle the high-dimensional decision space and dynamic nature of these networks. This paper proposes a novel hierarchical deep reinforcement learning (HDRL) framework to address these challenges and enable intelligent spectrum sharing. The proposed framework leverages the inherent hierarchy of the network, with separate policies for each tier, to learn and optimize spectrum allocation decisions at different timescales and levels of abstraction. By decomposing the complex spectrum sharing problem into manageable sub-tasks and allowing for efficient coordination among the tiers, the HDRL approach offers a scalable and adaptive solution for spectrum management in future TN-NTNs. Simulation results demonstrate the superior performance of the proposed framework compared to traditional approaches, highlighting its potential to enhance spectral efficiency and network capacity in dynamic, multi-tier environments.

Problem

• Addresses spectrum sharing in integrated TN-NTNs
• Manages interference in multi-tier, heterogeneous networks
• Optimizes spectrum allocation using hierarchical deep reinforcement learning

Innovation

• Hierarchical deep reinforcement learning framework
• Separate policies for each network tier
• Scalable and adaptive spectrum management solution