Parametrized Sharing for Multi-Agent Hybrid DRL for Multiple Multi-Functional RISs-Aided Downlink NOMA Networks

2026-01-02

AI Summary

This study addresses the energy efficiency maximization problem in a downlink non-orthogonal multiple access (NOMA) network assisted by multi-functional reconfigurable intelligent surfaces (MF-RISs), subject to constraints on transmit power, user data rates, and self-sustainable energy harvesting. A unified optimization framework is developed by jointly optimizing base station power allocation, beamforming, MF-RIS amplitude and phase shifts, energy harvesting ratios, and deployment locations. To solve this mixed-variable problem efficiently, the authors propose a parameter-sharing multi-agent hybrid deep reinforcement learning (PMHRL) approach that integrates proximal policy optimization (PPO) for continuous variables and deep Q-networks (DQN) for discrete decisions. This work pioneers the integration of multiple MF-RISs into NOMA downlink systems, simultaneously enabling signal manipulation and energy harvesting. Experimental results demonstrate that the proposed method significantly outperforms baseline schemes—including non-shared architectures and pure PPO or DQN approaches—in terms of energy efficiency.

Technology Category

• Multiagent Systems: Multiagent Learning
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Multi-functional reconfigurable intelligent surface (MF-RIS) is conceived to address the communication efficiency thanks to its extended signal coverage from its active RIS capability and self-sustainability from energy harvesting (EH). We investigate the architecture of multi-MF-RISs to assist non-orthogonal multiple access (NOMA) downlink networks. We formulate an energy efficiency (EE) maximization problem by optimizing power allocation, transmit beamforming and MF-RIS configurations of amplitudes, phase-shifts and EH ratios, as well as the position of MF-RISs, while satisfying constraints of available power, user rate requirements, and self-sustainability property. We design a parametrized sharing scheme for multi-agent hybrid deep reinforcement learning (PMHRL), where the multi-agent proximal policy optimization (PPO) and deep-Q network (DQN) handle continuous and discrete variables, respectively. The simulation results have demonstrated that proposed PMHRL has the highest EE compared to other benchmarks, including cases without parametrized sharing, pure PPO and DQN. Moreover, the proposed multi-MF-RISs-aided downlink NOMA achieves the highest EE compared to scenarios of no-EH/amplification, traditional RISs, and deployment without RISs/MF-RISs under different multiple access.

Problem

• Reconfigurable Intelligent Surface
• Non-Orthogonal Multiple Access
• Energy Efficiency
• Multi-Agent Reinforcement Learning
• Energy Harvesting

Innovation

• Parametrized Sharing
• Multi-Agent Hybrid DRL
• Multi-Functional RIS
• NOMA
• Energy Efficiency