Flex-MIG: Enabling Distributed Execution on MIG

2025-11-12

AI Summary

In multi-tenant GPU clusters, NVIDIA’s Multi-Instance GPU (MIG) technology suffers from severe resource fragmentation and low utilization due to its hardware-enforced rigidity and strict one-to-one instance-to-workload allocation model. This paper proposes Flex-MIG, the first software-only framework enabling collective communication over host-shared memory across MIG instances—thereby circumventing hardware-enforced isolation without any hardware modification. Flex-MIG integrates software-defined scheduling, cross-instance shared memory management, and distributed execution coordination to support flexible one-to-many resource allocation and low-overhead dynamic reconfiguration. Evaluated under diverse workloads, Flex-MIG reduces fragmentation by up to 42% and improves aggregate GPU utilization by up to 31% compared to baseline MIG deployments. Furthermore, it shortens job completion time by up to 17%, demonstrating substantial gains in cluster efficiency and responsiveness.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Multiagent Systems: Coordination and Collaboration

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: The sharing economy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

GPU clusters in multi-tenant settings often suffer from underutilization, making GPU-sharing technologies essential for efficient resource use. Among them, NVIDIA Multi-Instance GPU (MIG) has gained traction for providing hardware-level isolation that enables concurrent workloads without interference. However, MIG's hardware rigidity and the conventional one-to-one allocation model jointly lead to severe fragmentation and cluster-wide underutilization. We present Flex-MIG, a software-only framework that replaces one-to-one with a one-to-many allocation model and enables host-shared-memory collectives across MIG instances without hardware modification. Flex-MIG eliminates drain-required reconfiguration, reduces fragmentation, and improves makespan by up to 17% across diverse traces, showing that rethinking MIG's operational model as a software-coordinated layer substantially improves cluster efficiency.

Problem

• Addresses GPU cluster underutilization in multi-tenant environments
• Solves MIG's hardware rigidity causing fragmentation and inefficiency
• Replaces rigid one-to-one allocation with flexible one-to-many model

Innovation

• Replaces one-to-one with one-to-many allocation model
• Enables shared-memory collectives across MIG instances
• Software-only framework without hardware modification