Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System

2026-07-03

AI Summary

This work addresses the challenge of ensuring safety in industrial cyber-physical systems when applying deep reinforcement learning, where black-box exploration may inadvertently violate hardware constraints and conventional reward shaping struggles to balance safety with task performance. To overcome this, the authors propose a physics-informed safety mechanism that embeds a differentiable dynamics model into the loss function of a Proximal Policy Optimization (PPO) policy network. By performing short-horizon forward simulations to predict trajectories, the method imposes soft penalties—decoupled from the task-specific reward—on potential safety violations. This approach regularizes the policy online without requiring intricate reward engineering. Evaluated on a one-degree-of-freedom helicopter simulation platform, the method significantly reduces pitch angle constraint violations while maintaining excellent trajectory tracking performance.

Abstract

Deep reinforcement learning (DRL) offers powerful control for industrial cyber-physical systems (ICPSs), but its "black-box" exploration risks violating strict hardware safety limits. Typically, these constraints are managed through complex reward shaping. In this work-in-progress paper, we embed a differentiable physics model directly into the proximal policy optimization (PPO) actor loss function. By simulating short-horizon future trajectories during training, the policy is penalized for anticipated safety violations independent of the task-reward signal. Evaluated on a simulated 1-degree-of-freedom helicopter testbed with strict pitch constraints, our physics-informed soft regularizations substantially reduce constraint violations while maintaining reliable target tracking.

Problem

• Safe Reinforcement Learning
• Physics-Informed Neural Networks
• Hardware Safety Constraints
• Cyber-Physical Systems
• Constraint Violations

Innovation

• Physics-Informed Neural Networks
• Safe Reinforcement Learning
• Proximal Policy Optimization
• Differentiable Physics Model
• Constraint Violation