Prompt-Tuned LLM-Augmented DRL for Dynamic O-RAN Network Slicing

2025-05-31

AI Summary

In O-RAN dynamic network slicing, sparse and time-varying feedback hinders effective policy learning in conventional deep reinforcement learning (DRL) frameworks. Method: This paper proposes PA-MRL—a Prompt-Augmented Multi-agent Reinforcement Learning framework—that introduces a novel prompt-driven, LLM-based state contextualization mechanism. It employs learnable soft prompts to adapt a domain-specific LLM (ORANSight), enabling semantic-enriched state representation without full-model fine-tuning. Furthermore, it jointly optimizes semantic clustering and multi-agent RL objectives to enhance decision interpretability and low-overhead adaptability. Results: Experiments demonstrate that PA-MRL reduces training iterations by over 40%, significantly improves resource allocation adaptability, and achieves state-of-the-art performance across key metrics: throughput, end-to-end latency, and network slice SLA compliance rate.

Technology Category

• Machine Learning: Reinforcement Learning
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Modern wireless networks must adapt to dynamic conditions while efficiently managing diverse service demands. Traditional deep reinforcement learning (DRL) struggles in these environments, as scattered and evolving feedback makes optimal decision-making challenging. Large Language Models (LLMs) offer a solution by structuring unorganized network feedback into meaningful latent representations, helping RL agents recognize patterns more effectively. For example, in O-RAN slicing, concepts like SNR, power levels and throughput are semantically related, and LLMs can naturally cluster them, providing a more interpretable state representation. To leverage this capability, we introduce a contextualization-based adaptation method that integrates learnable prompts into an LLM-augmented DRL framework. Instead of relying on full model fine-tuning, we refine state representations through task-specific prompts that dynamically adjust to network conditions. Utilizing ORANSight, an LLM trained on O-RAN knowledge, we develop Prompt-Augmented Multi agent RL (PA-MRL) framework. Learnable prompts optimize both semantic clustering and RL objectives, allowing RL agents to achieve higher rewards in fewer iterations and adapt more efficiently. By incorporating prompt-augmented learning, our approach enables faster, more scalable, and adaptive resource allocation in O-RAN slicing. Experimental results show that it accelerates convergence and outperforms other baselines.

Problem

• Adapting wireless networks to dynamic conditions efficiently
• Structuring unorganized network feedback using LLMs for DRL
• Optimizing O-RAN slicing with prompt-augmented multi-agent RL

Innovation

• LLM-augmented DRL for dynamic O-RAN slicing
• Learnable prompts optimize semantic clustering
• Prompt-Augmented Multi-agent RL framework