Topology-Aware Graph Reinforcement Learning for Energy Storage Systems Optimal Dispatch in Distribution Networks

2026-03-27

AI Summary

This study addresses the challenge of co-optimizing economic efficiency and voltage security in energy storage dispatch for distribution networks under time-varying operating conditions and topology changes. The authors propose a Twin Delayed Deep Deterministic Policy Gradient (TD3) reinforcement learning framework integrated with graph neural networks (GNNs). This approach pioneers the application of topology-aware graph-based reinforcement learning to energy storage scheduling, incorporating three graph encoders—GCN, TAGConv, and GAT—to capture structural grid features and enable rapid online decision-making. Experiments on 34- and 69-node test systems demonstrate that the proposed GNN-enhanced controllers significantly reduce both the frequency and magnitude of voltage violations. Notably, TD3-GCN and TD3-TAGConv outperform conventional neural network-based methods in the 69-node system, achieving lower operational costs. Zero-shot cross-system transfer performance is shown to depend critically on scenario similarity, with substantial topological discrepancies leading to degraded efficacy.

Abstract

Optimal dispatch of energy storage systems (ESSs) in distribution networks involves jointly improving operating economy and voltage security under time-varying conditions and possible topology changes. To support fast online decision making, we develop a topology-aware Reinforcement Learning architecture based on Twin Delayed Deep Deterministic Policy Gradient (TD3), which integrates graph neural networks (GNNs) as graph feature encoders for ESS dispatch. We conduct a systematic investigation of three GNN variants: graph convolutional networks (GCNs), topology adaptive graph convolutional networks (TAGConv), and graph attention networks (GATs) on the 34-bus and 69-bus systems, and evaluate robustness under multiple topology reconfiguration cases as well as cross-system transfer between networks with different system sizes. Results show that GNN-based controllers consistently reduce the number and magnitude of voltage violations, with clearer benefits on the 69-bus system and under reconfiguration; on the 69-bus system, TD3-GCN and TD3-TAGConv also achieve lower saved cost relative to the NLP benchmark than the NN baseline. We also highlight that transfer gains are case-dependent, and zero-shot transfer between fundamentally different systems results in notable performance degradation and increased voltage magnitude violations. This work is available at: https://github.com/ShuyiGao/GNNs_RL_ESSs and https://github.com/distributionnetworksTUDelft/GNNs_RL_ESSs.

Problem

• energy storage systems
• optimal dispatch
• distribution networks
• voltage security
• topology changes

Innovation

• Graph Neural Networks
• Reinforcement Learning
• Energy Storage Systems
• Distribution Network
• Topology-aware Control