Data-driven policy mapping for safe RL-based energy management systems

2025-06-01

Energy Reports

AI Summary

To address the security–performance trade-off in deploying reinforcement learning (RL) policies within building energy management systems (BEMS), this paper proposes a data-driven, lightweight policy mapping framework. Unlike conventional approaches, it preserves the original RL policy intact and achieves online safety-constraint embedding and real-time regulatory compliance via local linearization of the observation space and backward mapping of safety boundaries. The framework integrates Lyapunov-guided safety verification, local affine approximation, online policy projection, and historical-data-driven boundary learning. Evaluated on a microgrid simulation and hardware-in-the-loop testbed, the method achieves 100% avoidance of safety violations, incurs an average energy-efficiency degradation of less than 1.2%, and accelerates safety response by 3.8× over baselines including CPO and Safe-DDPG. To the best of our knowledge, this is the first work to jointly optimize high safety assurance and minimal performance loss in BEMS RL deployment.

Technology Category

• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Security and privacy of machine learning and AI applications

Problem

• Address scalability and adaptability in energy management systems
• Ensure safe exploration and operation in RL-based systems
• Reduce operating costs and maintain environmental performance

Innovation

• Clustering load profiles for policy generalization
• LSTM forecasting for dynamic condition responsiveness
• Domain-informed action masking for safe operation