Toward an Energy-Optimized Operation of Data Centers Located in Wind Farms Using Reinforcement Learning

2026-06-29

AI Summary

This study addresses the challenge of online energy consumption optimization for data centers within wind farms under uncertainty, where future wind availability and electricity prices are unknown. To enhance the utilization of free wind energy and alleviate the credit assignment problem, the authors propose a reinforcement learning–based scheduling approach that integrates imitation learning with potential-based reward shaping. The method combines Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) algorithms equipped with an online update mechanism, enabling wind curtailment–aware workload migration within a reproducible fixed-day simulation framework. Experimental results over a 200-day test set demonstrate that the proposed approach significantly outperforms baseline reinforcement learning strategies. Although it slightly underperforms compared to the offline optimal solution with full foresight, it establishes a robust foundation for extending to multi-site and continuous-time scenarios.

Abstract

This paper studies Reinforcement Learning as an online controller for curtailment-aware workload shifting in wind-turbine-integrated high-performance computing (HPC) data centers. We introduce a reproducible fixed-day simulation framework with synthetic wind and price signals and delayed completion feedback, designed to be extensible toward more complex scenarios. As a controlled benchmarking basis, we then focus on the minimal case with one wind turbine and one co-located data center. In this setting, pure Reinforcement Learning exhibits a pronounced credit-assignment problem and tends to underuse free wind energy early in the day. We therefore evaluate two complementary countermeasures: optimization-based Imitation Learning and potential-based Reward Shaping. Across multi-seed training and a 200-day test set, Proximal Policy Optimization (PPO) and a Soft Actor-Critic (SAC) variant with an additional on-policy update routine achieve strong empirical performance among learned policies, and both Imitation Learning and Reward Shaping provide improvements in relevant configurations. A performance gap to the optimizer remains, which is expected: the optimizer plans offline with full-day foresight, whereas Reinforcement Learning must decide online from current observations without future realizations. The benchmark and ablation results provide a transparent basis for extending the approach toward richer multi-site and continuous-time scenarios.

Problem

• data centers
• wind energy
• workload shifting
• energy optimization
• reinforcement learning

Innovation

• Reinforcement Learning
• Imitation Learning
• Reward Shaping
• Energy Optimization
• Data Center Curtailment