Meta Hierarchical Reinforcement Learning for Scalable Resource Management in O-RAN

2025-12-08

AI Summary

Dynamic resource management and joint optimization of multi-service network slicing in O-RAN face significant challenges, particularly under highly time-varying conditions where conventional AI methods suffer from poor generalization. Method: This paper proposes the first meta-hierarchical reinforcement learning (meta-HRL) framework tailored for eMBB, URLLC, and mMTC services. It integrates a two-tier architecture—inter-slice allocation at the meta-level and intra-slice scheduling at the base level—and introduces a TD-error variance-weighted adaptive meta-update mechanism, with theoretical guarantees on sublinear convergence and regret bounds. Contribution/Results: It is the first to realize unified, online, and adaptive multi-slice control within the RIC architecture. Experiments demonstrate a 19.8% improvement in network management efficiency over baseline RL and meta-RL approaches, significantly higher QoS satisfaction rates, 40% faster adaptation in large-scale deployments, and stable fairness, latency, and throughput performance.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Hardware-aware ML
• Multiagent Systems: Multiagent Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The increasing complexity of modern applications demands wireless networks capable of real time adaptability and efficient resource management. The Open Radio Access Network (O-RAN) architecture, with its RAN Intelligent Controller (RIC) modules, has emerged as a pivotal solution for dynamic resource management and network slicing. While artificial intelligence (AI) driven methods have shown promise, most approaches struggle to maintain performance under unpredictable and highly dynamic conditions. This paper proposes an adaptive Meta Hierarchical Reinforcement Learning (Meta-HRL) framework, inspired by Model Agnostic Meta Learning (MAML), to jointly optimize resource allocation and network slicing in O-RAN. The framework integrates hierarchical control with meta learning to enable both global and local adaptation: the high-level controller allocates resources across slices, while low level agents perform intra slice scheduling. The adaptive meta-update mechanism weights tasks by temporal difference error variance, improving stability and prioritizing complex network scenarios. Theoretical analysis establishes sublinear convergence and regret guarantees for the two-level learning process. Simulation results demonstrate a 19.8% improvement in network management efficiency compared with baseline RL and meta-RL approaches, along with faster adaptation and higher QoS satisfaction across eMBB, URLLC, and mMTC slices. Additional ablation and scalability studies confirm the method's robustness, achieving up to 40% faster adaptation and consistent fairness, latency, and throughput performance as network scale increases.

Problem

• Optimizes resource allocation and network slicing in O-RAN under dynamic conditions
• Enables global and local adaptation through hierarchical control with meta-learning
• Improves network management efficiency and QoS across diverse service slices

Innovation

• Meta-HRL framework for O-RAN resource optimization
• Hierarchical control with meta-learning for global-local adaptation
• Adaptive meta-update weighted by temporal difference error variance