ORAN-GUIDE: RAG-Driven Prompt Learning for LLM-Augmented Reinforcement Learning in O-RAN Network Slicing

2025-05-31

AI Summary

To address weak policy generalization and slow convergence in dynamic, heterogeneous service scheduling for O-RAN network slicing—under partially observable, unstructured inputs (e.g., RF features, QoS metrics, traffic trends)—this paper proposes a dual-LLM-enhanced multi-agent reinforcement learning (MARL) framework. We introduce a RAG-driven prompt learning paradigm: a domain-specific language model, ORANSight, generates semantically enriched prompts; these are fused with learnable tokens and a frozen large language model encoder to achieve expressive state abstraction. This work is the first to jointly leverage a domain-pretrained LLM and a frozen LLM for MARL state representation, enabling semantic-aware policy generalization. Experiments demonstrate that our method improves average QoS assurance rate by 18.7% and cross-scenario generalization performance by 32.4% over standard MARL and single-LLM baselines, while significantly enhancing sample efficiency and convergence speed.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Advanced wireless networks must support highly dynamic and heterogeneous service demands. Open Radio Access Network (O-RAN) architecture enables this flexibility by adopting modular, disaggregated components, such as the RAN Intelligent Controller (RIC), Centralized Unit (CU), and Distributed Unit (DU), that can support intelligent control via machine learning (ML). While deep reinforcement learning (DRL) is a powerful tool for managing dynamic resource allocation and slicing, it often struggles to process raw, unstructured input like RF features, QoS metrics, and traffic trends. These limitations hinder policy generalization and decision efficiency in partially observable and evolving environments. To address this, we propose extit{ORAN-GUIDE}, a dual-LLM framework that enhances multi-agent RL (MARL) with task-relevant, semantically enriched state representations. The architecture employs a domain-specific language model, ORANSight, pretrained on O-RAN control and configuration data, to generate structured, context-aware prompts. These prompts are fused with learnable tokens and passed to a frozen GPT-based encoder that outputs high-level semantic representations for DRL agents. This design adopts a retrieval-augmented generation (RAG) style pipeline tailored for technical decision-making in wireless systems. Experimental results show that ORAN-GUIDE improves sample efficiency, policy convergence, and performance generalization over standard MARL and single-LLM baselines.

Problem

• Enhancing dynamic resource allocation in O-RAN network slicing
• Improving policy generalization in partially observable environments
• Processing unstructured input like RF features and QoS metrics

Innovation

• Dual-LLM framework enhances MARL with semantic states
• Domain-specific ORANSight generates structured context-aware prompts
• RAG pipeline improves DRL decision-making in wireless systems