Deep Reinforcement Learning-Empowered Wireless Sensor Networking for 6G Closed-Loop Controls

2026-07-09

AI Summary

This work addresses the degradation of control performance in bandwidth-constrained 6G sensing–communication–computation–control (SC3) closed-loop systems by proposing a control-performance-driven intelligent bandwidth allocation scheme. The closed-loop process is modeled as a partially observable Markov decision process (POMDP), where mutual information theory quantifies the state estimation distortion induced by communication rate limitations. An optimization objective is formulated by integrating Kalman filtering with linear quadratic regulator (LQR) control. Innovatively combining deep reinforcement learning (DRL) and information theory, the approach employs proximal policy optimization (PPO) to dynamically allocate bandwidth from sensors to edge information hubs. Simulations demonstrate that the proposed method significantly outperforms conventional static or heuristic strategies, effectively reducing LQR control cost and enhancing overall system performance under limited communication resources.

Abstract

Robots are increasingly deployed in remote or hazardous areas for mission-critical control tasks. Due to their limited individual capabilities, they have to rely on other field sensors to obtain the state information of targets, and also a dedicated edge information hub (EIH) to enable information exchange, sensing data analysis and control command generation. Such configuration follows a sensing-communication-computing-control (SC3) closed loop. To optimize the whole closed-loop performance, this paper minimizes the linear quadratic regulator (LQR) control cost by designing the sensor-to-EIH bandwidth allocation. Specifically, we first model the distortion noise caused by limited communication data rate based on the mutual information theory. Next, under the control policy based on the Kalman filter and LQR controller, we formulate the control process as a partially observable Markov decision process (POMDP), and develop a deep reinforcement learning (DRL)-based sensor-to-EIH bandwidth allocation scheme. The proximal policy optimization (PPO) algorithm is utilized to train the DRL agent. Simulation results are provided to show the superiority of the proposed DRL-based scheme.

Problem

• Wireless Sensor Networking
• Closed-Loop Control
• Bandwidth Allocation
• LQR Control Cost
• 6G

Innovation

• Deep Reinforcement Learning
• Bandwidth Allocation
• Closed-Loop Control
• POMDP
• 6G Wireless Sensor Networks