SMART-MIG: A Learning Framework for Scalable and Energy-Efficient GPU Scheduling

2026-06-29

AI Summary

This work addresses the joint optimization of energy consumption and task latency in large-scale GPU scheduling, compounded by the dual complexities of dynamic Multi-Instance GPU (MIG) partitioning and job scheduling. To tackle this challenge, the authors propose a cooperative scheduling framework that integrates mean-field multi-agent reinforcement learning (MF-MARL) with a tailored heuristic algorithm. A key innovation is a scalable dynamic MIG repartitioning mechanism whose computational complexity is independent of both task count and GPU scale. Furthermore, the study establishes, for the first time, a theoretical lower bound on the energy-latency trade-off to serve as a performance benchmark. Experimental results demonstrate that the proposed approach improves overall energy-delay performance by 18% over static partitioning schemes, while achieving total energy consumption only 27% above the theoretical lower bound.

Abstract

The emergence of Multi-Instance GPU (MIG) technology enables us to run smaller machine learning models on partitions of a GPU rather than the entire device, thus improving utilization and reducing energy consumption, albeit with potential performance trade-offs. Meanwhile, the growing energy demands of GPU-equipped data centers motivate the development of online partitioning and scheduling schemes that not only ensure fast job processing but also achieve high energy efficiency. However, achieving energy-tardiness efficiency with manageable algorithmic complexity in large-scale scheduling remains a great challenge, due to the dual objectives of deciding on the GPU partitions and scheduling jobs onto the slices of the heterogeneous partitions. To address this challenge, we propose SMART-MIG, a parallel computing system that combines Mean-Field Multi-Agent Reinforcement Learning (MF-MARL) for large-scale MIG repartitioning with tailored heuristic algorithms for job scheduling. We demonstrate that the complexity of the repartitioning component remains constant even as the number of jobs and GPUs increases. We also establish theoretical lower bounds on energy consumption and tardiness to rigorously benchmark system performance. Finally, extensive experiments show that SMART-MIG improves the energy-tardiness efficiency by $18\%$ compared to its corresponding static-partitioning counterpart, while being only $27\%$ above the theoretical lower bound on energy consumption.

Problem

• Multi-Instance GPU
• energy efficiency
• job scheduling
• GPU partitioning
• scalable scheduling

Innovation

• Multi-Instance GPU
• Mean-Field Multi-Agent Reinforcement Learning
• Energy-Efficient Scheduling
• Scalable Partitioning
• Tardiness-Energy Trade-off