Improving the Efficiency of a Deep Reinforcement Learning-Based Power Management System for HPC Clusters Using Curriculum Learning

2025-02-27

AI Summary

High-performance computing (HPC) clusters suffer from high idle-energy consumption, and existing node power-on/off policies struggle to balance energy efficiency with quality-of-service (QoS) guarantees. To address this, we propose a deep reinforcement learning (DRL)-based power management framework incorporating curriculum learning (CL)—the first such integration in HPC power optimization. Our CL strategy designs a progressively challenging task sequence, significantly accelerating policy convergence and enhancing generalization across diverse cluster scales and hardware configurations without retraining. Implemented using the Proximal Policy Optimization (PPO) algorithm within the Batsim-py simulation framework, our method achieves 3.73% greater energy savings than baseline DRL approaches and outperforms the optimal fixed-timeout policy by 4.66%. It also reduces average job waiting time by 9.24%, improves job packing efficiency, and demonstrates robustness under multi-parameter sensitivity analysis.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

High energy consumption remains a key challenge in high-performance computing (HPC) systems, which often feature hundreds or thousands of nodes drawing substantial power even in idle or standby modes. Although powering down unused nodes can improve energy efficiency, choosing the wrong time to do so can degrade quality of service by delaying job execution. Machine learning, in particular reinforcement learning (RL), has shown promise in determining optimal times to switch nodes on or off. In this study, we enhance the performance of a deep reinforcement learning (DRL) agent for HPC power management by integrating curriculum learning (CL), a training approach that introduces tasks with gradually increasing difficulty. Using the Batsim-py simulation framework, we compare the proposed CL-based agent to both a baseline DRL method (without CL) and the conventional fixed-time timeout strategy. Experimental results confirm that an easy-to-hard curriculum outperforms other training orders in terms of reducing wasted energy usage. The best agent achieves a 3.73% energy reduction over the baseline DRL method and a 4.66% improvement compared to the best timeout configuration (shutdown every 15 minutes of idle time). In addition, it reduces average job waiting time by 9.24% and maintains a higher job-filling rate, indicating more effective resource utilization. Sensitivity tests across various switch-on durations, power levels, and cluster sizes further reveal the agent's adaptability to changing system parameters without retraining. These findings demonstrate that curriculum learning can significantly improve DRL-based power management in HPC, balancing energy savings, quality of service, and robustness to diverse configurations.

Problem

• Optimize HPC power management efficiency
• Reduce energy consumption using curriculum learning
• Enhance job execution and resource utilization

Innovation

• Curriculum Learning for DRL
• Batsim-py Simulation Framework
• Energy Reduction in HPC