Deep Reinforcement Learning-Based Precoding for Multi-RIS-Aided Multiuser Downlink Systems with Practical Phase Shift

2025-09-29

AI Summary

This paper addresses the joint optimization of transmit precoding and reconfigurable intelligent surface (RIS) phase shifts in multi-RIS-aided multiuser downlink systems to maximize spectral efficiency, while introducing— for the first time in multi-RIS scenarios—a practical coupling constraint between reflection amplitude and phase. Method: To tackle non-convexity, channel time-varying dynamics, and stochastic user distribution, we propose an end-to-end joint optimization framework based on deep deterministic policy gradient (DDPG), enabling real-time adaptive design under millimeter-wave channels. Contribution/Results: Experiments demonstrate that the proposed method significantly outperforms conventional iterative algorithms and double deep Q-networks (double DQN). It maintains robust high performance under dynamic user counts, validating both the realism of the amplitude-phase coupling model and the efficacy and practicality of the reinforcement learning framework.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB
• Multiagent Systems: Multiagent Learning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This study considers multiple reconfigurable intelligent surfaces (RISs)-aided multiuser downlink systems with the goal of jointly optimizing the transmitter precoding and RIS phase shift matrix to maximize spectrum efficiency. Unlike prior work that assumed ideal RIS reflectivity, a practical coupling effect is considered between reflecting amplitude and phase shift for the RIS elements. This makes the optimization problem non-convex. To address this challenge, we propose a deep deterministic policy gradient (DDPG)-based deep reinforcement learning (DRL) framework. The proposed model is evaluated under both fixed and random numbers of users in practical mmWave channel settings. Simulation results demonstrate that, despite its complexity, the proposed DDPG approach significantly outperforms optimization-based algorithms and double deep Q-learning, particularly in scenarios with random user distributions.

Problem

• Optimizing precoding and RIS phase shifts for spectrum efficiency
• Addressing practical RIS amplitude-phase coupling effects
• Solving non-convex optimization in multi-RIS multiuser systems

Innovation

• Deep reinforcement learning optimizes precoding and phase shifts
• Addresses practical RIS element coupling effects
• Outperforms optimization-based and Q-learning algorithms