Reinforcement Learning-Based Secure Beamforming Against Satellite Eavesdroppers

2026-06-18

AI Summary

This work addresses physical-layer security in low Earth orbit satellite uplink communications under multiple non-colluding eavesdroppers by proposing a secrecy rate maximization method subject to an average outage constraint. Exploiting the predictability of orbital dynamics, the problem is formulated as a constrained Markov decision process. A primal-dual soft actor-critic reinforcement learning algorithm is developed for beamforming, incorporating a differentiable upper bound on the outage probability as the cost function and employing a multi-head cost critic to enable low-complexity real-time optimization. Theoretical analysis leverages closed-form outage probability expressions under Nakagami-m fading, with constraints handled via Lagrangian relaxation. Experimental results demonstrate that the proposed approach consistently outperforms maximal ratio transmission across diverse eavesdropping configurations and surpasses zero-forcing beamforming in dense eavesdropper scenarios, achieving 93% of the secrecy rate attained by an offline successive convex approximation benchmark with only a single forward computation.

Abstract

This paper investigates physical-layer security for uplink low Earth orbit (LEO) satellite communications in the presence of multiple non-colluding satellite eavesdroppers. Secure beamforming design in such systems is challenging due to time-varying orbital geometry and probabilistic fading-induced outage constraints. To address this, we first derive tractable closed-form expressions for both connection and secrecy outage probabilities under Nakagami-m fading, and develop differentiable upper-bound cost functions that are amenable to optimization. Next, to exploit the predictable orbital dynamics and temporal correlation of satellite mobility, we reformulate the non-convex secrecy rate maximization problem as a constrained Markov decision process. We then develop a primal-dual soft actor-critic algorithm with a multi-head cost critic that jointly optimizes beamforming while enforcing average outage constraints via Lagrangian relaxation. Numerical results show that the proposed framework improves the ergodic secrecy rate over maximum ratio transmission across all eavesdropper configurations, and outperforms zero-forcing in dense eavesdropping regimes. It achieves within 7 percent of an offline successive convex approximation benchmark while requiring only a single forward pass, enabling low-complexity real-time operation. These results indicate that the proposed approach is applicable to secure beamforming in dynamic LEO satellite environments.

Problem

• secure beamforming
• satellite eavesdroppers
• physical-layer security
• LEO satellite communications
• outage constraints

Innovation

• secure beamforming
• reinforcement learning
• LEO satellite communications
• physical-layer security
• constrained Markov decision process