CommonPower: A Framework for Safe Data-Driven Smart Grid Control

2024-06-05

AI Summary

Reinforcement learning (RL) controllers in smart grids lack formal safety guarantees and struggle to simultaneously integrate predictive models and support verifiable decentralized control. Method: This paper proposes CommonPower—a novel framework featuring symbolic modeling to unify single- and multi-agent RL with model predictive control (MPC), enabling co-design, safety verification, and joint optimization. It provides formally verifiable safety enforcement mechanisms and seamless interfaces between RL controllers and load/renewable generation forecasts. Contribution/Results: Implemented as a Python-based toolchain, CommonPower integrates symbolic modeling, multi-agent RL, and MPC. Evaluated on a building energy management case study, it demonstrates the efficacy of multiple safety-certified RL controllers and conducts a systematic performance comparison against MPC. The framework significantly enhances the safety assurance, formal verifiability, and practical deployability of smart grid control systems.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The growing complexity of power system management has led to an increased interest in reinforcement learning (RL). However, vanilla RL controllers cannot themselves ensure satisfaction of system constraints. Therefore, combining them with formally correct safeguarding mechanisms is an important aspect when studying RL for power system management. Integrating safeguarding into complex use cases requires tool support. To address this need, we introduce the Python tool CommonPower. CommonPower's unique contribution lies in its symbolic modeling approach, which enables flexible, model-based safeguarding of RL controllers. Moreover, CommonPower offers a unified interface for single-agent RL, multi-agent RL, and optimal control, with seamless integration of different forecasting methods. This allows users to validate the effectiveness of safe RL controllers across a large variety of case studies and investigate the influence of specific aspects on overall performance. We demonstrate CommonPower's versatility through a numerical case study that compares RL agents featuring different safeguards with a model predictive controller in the context of building energy management.

Problem

• Addresses complexity in power system management using reinforcement learning.
• Provides a framework for safe, data-driven smart grid control.
• Automates synthesis of safeguards and model predictive controllers.

Innovation

• Modular Python framework for power system simulation
• Automatic synthesis of predictive controllers and safeguards
• Unified interface for RL, multi-agent RL, and optimal control