Task Specific Sharpness Aware O-RAN Resource Management using Multi Agent Reinforcement Learning

2025-11-19

AI Summary

To address the insufficient robustness and generalization of deep reinforcement learning (DRL) models in dynamic resource management under O-RAN architectures, this paper proposes a distributed multi-agent reinforcement learning (MARL)-based resource allocation framework. Methodologically, it integrates Soft Actor-Critic (SAC) with Sharpness-Aware Minimization (SAM) to enhance model stability and adaptability in heterogeneous, time-varying O-RAN environments. Key contributions include: (1) an adaptive decision-making mechanism conditioned on task specificity and environmental complexity; (2) a differentiable, TD-error-variance-triggered SAM regularization for on-demand model stabilization; and (3) a dynamic ρ-scheduling strategy that jointly optimizes exploration–exploitation trade-offs. Experimental results demonstrate that the proposed approach achieves up to 22% higher resource allocation efficiency and significantly improved QoS satisfaction rates over baseline methods, validating its effectiveness and practicality in real-world O-RAN deployments.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Next-generation networks utilize the Open Radio Access Network (O-RAN) architecture to enable dynamic resource management, facilitated by the RAN Intelligent Controller (RIC). While deep reinforcement learning (DRL) models show promise in optimizing network resources, they often struggle with robustness and generalizability in dynamic environments. This paper introduces a novel resource management approach that enhances the Soft Actor Critic (SAC) algorithm with Sharpness-Aware Minimization (SAM) in a distributed Multi-Agent RL (MARL) framework. Our method introduces an adaptive and selective SAM mechanism, where regularization is explicitly driven by temporal-difference (TD)-error variance, ensuring that only agents facing high environmental complexity are regularized. This targeted strategy reduces unnecessary overhead, improves training stability, and enhances generalization without sacrificing learning efficiency. We further incorporate a dynamic $ ho$ scheduling scheme to refine the exploration-exploitation trade-off across agents. Experimental results show our method significantly outperforms conventional DRL approaches, yielding up to a $22%$ improvement in resource allocation efficiency and ensuring superior QoS satisfaction across diverse O-RAN slices.

Problem

• Enhancing robustness and generalizability of DRL models in dynamic O-RAN environments
• Optimizing resource allocation efficiency and QoS satisfaction across network slices
• Reducing training overhead while maintaining learning stability in multi-agent systems

Innovation

• Multi-Agent RL with Sharpness-Aware Minimization enhancement
• Adaptive SAM regularization driven by TD-error variance
• Dynamic rho scheduling for exploration-exploitation trade-off