Hierarchical Multi-Agent Reinforcement Learning for Carbon-Aware AI Data Centers in Power Distribution Systems

2026-07-03

AI Summary

This study addresses the escalating carbon emissions associated with surging energy consumption in AI data centers by proposing a carbon-aware hierarchical multi-agent reinforcement learning framework that jointly optimizes spatiotemporal task scheduling, GPU resource allocation, and cooling control. For the first time, the framework integrates real-time carbon intensity data from distribution network nodes into the decision-making process, leveraging a multi-agent Transformer architecture to co-model carbon emission flows, power distribution constraints, and thermal management dynamics. Experimental results on the IEEE 33-node distribution system demonstrate that the proposed approach significantly reduces carbon emissions while enhancing operational efficiency, effectively balancing economic performance with low-carbon objectives.

Abstract

Eco-friendly energy management for artificial intelligence data centers (AIDCs) is crucial because of the significant increase in energy consumption-induced carbon emissions from AIDCs resulting from the rapid expansion of AI applications. This paper proposes a hierarchical carbon-aware multi-agent reinforcement learning (CA-MARL) framework for robust and efficient operations of AIDCs under uncertainties while ensuring low-carbon operation of power distribution systems. The framework comprises a workload manager (WM) agent and multiple local AIDC agents trained using a multi-agent transformer method, corresponding to a global AIDC aggregator and a local AIDC operator, respectively. Leveraging AIDC operation data along with nodal carbon intensity (NCI) calculated from the carbon emission flow-integrated distribution system operator problem, the WM agent spatially allocates AI training and inference jobs among all AIDCs. Based on the jobs allocated from the WM agent and NCI information, each AIDC agent schedules economical and eco-friendly operations of the AIDC by performing the following tasks: i) temporal shifting of training jobs, ii) spatial allocation of training graphics processing unit (GPU) blocks and inference GPUs within the AIDC, and iii) control of the supply air temperature of the cooling system. The effectiveness of the proposed framework was assessed using an IEEE 33-node power distribution system.

Problem

• carbon-aware
• AI data centers
• power distribution systems
• energy management
• carbon emissions

Innovation

• carbon-aware
• multi-agent reinforcement learning
• hierarchical architecture
• nodal carbon intensity
• AI data centers