A Hierarchical Optimization Framework Using Deep Reinforcement Learning for Task-Driven Bandwidth Allocation in 5G Teleoperation

2025-05-21

AI Summary

To address the heterogeneous resource co-allocation challenge between URLLC control signaling and eMBB data transmission in 5G remote operation, this paper proposes a communication-control joint optimization framework. First, we formulate a network-slicing-enabled dual-queue delay-constrained model and integrate Lyapunov optimization with Lagrangian dual decomposition to provide theoretical guarantees for resource scheduling. Second, we introduce a novel DRL-driven two-layer architecture: an upper layer employs an Actor-Critic agent to dynamically balance eMBB/URLLC bandwidth allocation, while a lower layer performs real-time robotic control gain adaptation. Experimental results demonstrate that the proposed scheme reduces URLLC end-to-end delay jitter by 37% and achieves 99.999% reliability; simultaneously, it improves eMBB throughput by 22% and resource utilization by 31%, with measured closed-loop latency consistently maintained below 10 ms.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

The evolution of 5G wireless technology has revolutionized connectivity, enabling a diverse range of applications. Among these are critical use cases such as real time teleoperation, which demands ultra reliable low latency communications (URLLC) to ensure precise and uninterrupted control, and enhanced mobile broadband (eMBB) services, which cater to data-intensive applications requiring high throughput and bandwidth. In our scenario, there are two queues, one for eMBB users and one for URLLC users. In teleoperation tasks, control commands are received in the URLLC queue, where communication delays occur. The dynamic index (DI) controls the service rate, affecting the telerobotic (URLLC) queue. A separate queue models eMBB data traffic. Both queues are managed through network slicing and application delay constraints, leading to a unified Lagrangian-based Lyapunov optimization for efficient resource allocation. We propose a DRL based hierarchical optimization framework that consists of two levels. At the first level, network optimization dynamically allocates resources for eMBB and URLLC users using a Lagrangian functional and an actor critic network to balance competing objectives. At the second level, control optimization finetunes the best gains for robots, ensuring stability and responsiveness in network conditions. This hierarchical approach enhances both communication and control processes, ensuring efficient resource utilization and optimized performance across the network.

Problem

• Optimize bandwidth allocation for eMBB and URLLC in 5G teleoperation
• Balance competing objectives in network slicing with DRL framework
• Ensure stability and responsiveness in telerobotic control under delay constraints

Innovation

• Hierarchical DRL framework for 5G bandwidth allocation
• Lagrangian Lyapunov optimization for network slicing
• Actor-critic network balances eMBB and URLLC resources