Model-Free Reinforcement Learning Control for Resilient Cyber-Physical Systems

2026-06-17

AI Summary

This study addresses the resilience and performance challenges of cyber-physical systems under false data injection and denial-of-service attacks by proposing a model-free reinforcement learning control approach. The method incorporates a Lyapunov function–based reward mechanism and leverages both Proximal Policy Optimization (PPO) and Deep Deterministic Policy Gradient (DDPG) algorithms within RL-MPC and RL-PID control architectures. Experimental results demonstrate that the Lyapunov-based reward significantly enhances attack resilience and reduces tracking error, while PPO effectively lowers the variance of key performance metrics. Moreover, the RL-PID framework substantially shortens training time compared to RL-MPC. The work elucidates the trade-offs between different reward structures in balancing training efficiency and robustness, offering a novel paradigm for designing highly resilient intelligent control systems.

Abstract

This paper compares the performance of model-free controllers on a nonlinear system under cyberattacks, including false data injection and denial-of-service attacks. Four RL reward types are analyzed for accuracy, cost, and resilience. Results show that the Lyapunov reward offers the best resilience with low tracking error. Exponential mode also provides good trade-offs with acceptable resilience under moderate training conditions. Progressive and linear rewards converge faster but are less robust. RL-MPCs show strong steady-state resilience but require longer training times; RL-PID controllers are faster with significantly less training time. Proximal Policy Optimization outperforms Deep Deterministic Policy Gradient with a significant reduction in KPI variance. This study serves to highlight how well-designed RL rewards can improve performance and resilience against cyber threats.

Problem

• reinforcement learning
• cyber-physical systems
• cyberattacks
• resilience
• model-free control

Innovation

• Model-Free Reinforcement Learning
• Cyber-Physical Systems Resilience
• Reward Function Design
• Lyapunov-Based Reward
• Proximal Policy Optimization