Towards Affordable Energy: A Gymnasium Environment for Electric Utility Demand-Response Programs

2026-05-12

AI Summary

This work addresses the challenges of optimizing demand response at the distribution level and the financial risks faced by residential consumers under extreme weather events and volatile electricity prices. To this end, we propose DR-Gym—the first market-scale, online demand response simulation environment tailored for utility operators. Built upon the Gymnasium framework, DR-Gym integrates a state-transition electricity pricing model calibrated with real-world extreme-event data and a physics-informed building load model, while supporting configurable multi-objective reward structures. Experimental results demonstrate that DR-Gym generates realistic and learnable dynamic interaction scenarios, effectively enabling the training and evaluation of reinforcement learning–based demand response strategies and providing a reliable platform for algorithmic development in this domain.

Abstract

Extreme weather and volatile wholesale electricity markets expose residential consumers to catastrophic financial risks, yet demand response at the distribution level remains an underutilized tool for grid flexibility and energy affordability. While a demand-response program can shield consumers by issuing financial credits during high-price periods, optimizing this sequential decision-making process presents a unique challenge for reinforcement learning despite the plentiful offline historical smart meter and wholesale pricing data available publicly. Offline historical data fails to capture the dynamic, interactive feedback loop between an electric utility's pricing signals and customer acceptance and adaptation to a demand-response program. To address this, we introduce DR-Gym, an open-source, online Gymnasium-compatible environment designed to train and evaluate demand-response from the electric utility's perspective. Unlike existing device-level energy simulators, our environment focuses on the market-level electric utility setting and provides a rich observational space relevant to the electric utility. The simulator additionally features a regime-switching wholesale price model calibrated to real-world extreme events, alongside physics-based building demand profiles. For our learning signal, we use a configurable, multi-objective reward function for specifying diverse learning objectives. We demonstrate through baseline strategies and data snapshots the capability of our simulator to create realistic and learnable environments.

Problem

• demand response
• electric utility
• energy affordability
• grid flexibility
• wholesale electricity markets

Innovation

• DR-Gym
• demand response
• reinforcement learning
• wholesale electricity market
• Gymnasium-compatible environment