Addressing Terminal Constraints in Data-Driven Demand Response Scheduling

2026-05-14

AI Summary

This work addresses the challenges in data-driven demand response scheduling, where terminal constraints are difficult to satisfy over long time horizons, conventional model-based optimization incurs high computational costs, and reinforcement learning is often hindered by myopic behavior and credit assignment issues. To overcome these limitations, the paper proposes an approach that integrates Goal Space Planning (GSP) with Deep Deterministic Policy Gradient (DDPG). By constructing a temporally abstract model over discrete subgoals, the method effectively propagates long-horizon value signals, thereby alleviating credit assignment difficulties and enhancing adherence to terminal constraints. Experimental results on an air separation simulation benchmark demonstrate that the proposed method significantly outperforms standard DDPG, achieving substantial improvements in both sample efficiency and constraint satisfaction.

Abstract

Electrified chemical processes are incentivized by exposure to time-varying electricity markets to operate flexibly, but participating in demand response schemes can require satisfying terminal constraints over long horizons. Specifically, terminal constraints may be required when computing optimal schedules in order to preserve dynamic stability. Model-based optimization methods are computationally costly, and data-driven scheduling via reinforcement learning (RL) faces severe credit-assignment challenges. We integrate Goal-Space Planning (GSP) with Deep Deterministic Policy Gradient (DDPG), using learned temporally abstract models over discrete subgoals to propagate value across extended horizons. Using a simulated air separation benchmark, we demonstrate the proposed approach improves sample efficiency over standard DDPG while satisfying terminal storage constraints, mitigating myopic control behavior.

Problem

• terminal constraints
• demand response scheduling
• dynamic stability
• sample efficiency
• credit assignment

Innovation

• Goal-Space Planning
• Deep Deterministic Policy Gradient
• Terminal Constraints
• Sample Efficiency
• Demand Response Scheduling