Federated Deep Reinforcement Learning for Energy Efficient Multi-Functional RIS-Assisted Low-Earth Orbit Networks

2025-01-19

AI Summary

To address poor signal coverage and high energy consumption in shadowed regions of Low Earth Orbit (LEO) satellite networks, this paper proposes an on-orbit deployable multifunctional reconfigurable intelligent surface (MF-RIS) onboard satellites—first integrating reflection, refraction, active amplification, and radio-frequency (RF) energy harvesting capabilities. We jointly optimize MF-RIS configuration and satellite beamforming by designing a federated-enhanced multi-agent deep deterministic policy gradient (FEMAD) framework, enabling privacy-preserving distributed collaborative energy-efficiency optimization. Compared to centralized deep reinforcement learning (DRL) and distributed MADDPG baselines, the proposed method improves energy efficiency by over 35%. Relative to conventional reflective RIS, energy-harvesting–disabled MF-RIS, and RIS-free baselines, it achieves respective gains of 28%, 22%, and 41% in energy efficiency. These results significantly advance the green and sustainable evolution of LEO satellite networks.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In this paper, a novel network architecture that deploys the multi-functional reconfigurable intelligent surface (MF-RIS) in low-Earth orbit (LEO) is proposed. Unlike traditional RIS with only signal reflection capability, the MF-RIS can reflect, refract, and amplify signals, as well as harvest energy from wireless signals. Given the high energy demands in shadow regions where solar energy is unavailable, MF-RIS is deployed in LEO to enhance signal coverage and improve energy efficiency (EE). To address this, we formulate a long-term EE optimization problem by determining the optimal parameters for MF-RIS configurations, including amplification and phase-shifts, energy harvesting ratios, and LEO transmit beamforming. To address the complex non-convex and non-linear problem, a federated learning enhanced multi-agent deep deterministic policy gradient (FEMAD) scheme is designed. Multi-agent DDPG of each agent can provide the optimal action policy from its interaction to environments, whereas federated learning enables the hidden information exchange among multi-agents. In numerical results, we can observe significant EE improvements compared to the other benchmarks, including centralized deep reinforcement learning as well as distributed multi-agent deep deterministic policy gradient (DDPG). Additionally, the proposed LEO-MF-RIS architecture has demonstrated its effectiveness, achieving the highest EE performance compared to the scenarios of fixed/no energy harvesting in MF-RIS, traditional reflection-only RIS, and deployment without RISs/MF-RISs.

Problem

• Low Earth Orbit (LEO) Networks
• Multi-Functional Reconfigurable Intelligent Surface (MF-RIS)
• Signal Coverage and Energy Efficiency

Innovation

• MF-RIS
• Multi-agent Reinforcement Learning
• Energy Efficiency