DeepShare: Assurance-Driven Deep Learning Job Scheduling for Multi-Tenant Clusters

2026-09-15

AI Summary

DeepShare通过使用连续租户保证信号协调多租户GPU集群中的任务调度，结合多种策略提高GPU利用率并减少排队延迟。

Abstract

Multi-tenant GPU clusters frequently remain underutilized even when tenants experience long queueing delays, because quota control, queue ordering, preemption, and GPU sharing are driven by different local signals. We present DeepShare, a scheduler that uses a continuous tenant-assurance signal to coordinate these decisions at runtime. DeepShare combines elastic quota borrowing, tenant-specific runtime prediction, cost-aware best-effort preemption, and interference-aware MPS colocation, while using the same assurance signal to decide when borrowed capacity should be reclaimed and when sharing should become more conservative. In trace-driven experiments on 23,859 Venus jobs and 3,200 internal jobs, DeepShare achieves an average GPU utilization of 70.58%, a 29.5% improvement over the strongest non-intrusive sharing baseline, while reducing average queueing delay by 46%. On a 16-GPU Kubernetes testbed, it reduces the average job completion time by 34% and maintains 93% QoS compliance for guaranteed tenants. These results show that treating tenant assurance as a runtime control loop achieves a more advantageous utilization-QoS trade-off than optimizing quotas, scheduling, and resource sharing independently.

Problem

• multi-tenant GPU clusters
• underutilized
• queueing delays
• quota control
• preemption

Innovation

• tenant-assurance signal
• elastic quota borrowing
• cost-aware preemption
• interference-aware MPS colocation