Pareto-Optimal Scheduling in the Half-batch Multiserver-job Model

2026-07-09

AI Summary

This work addresses the fundamental challenge of simultaneously achieving low latency for large jobs and high throughput for small jobs in large-scale computing systems. The authors propose a semi-batch multi-server job (MSJ) model and introduce the convoy policy family along with its convex combinations to attain Pareto-optimal scheduling between these competing objectives. For the first time, they provide an exact, general, and non-asymptotic characterization of the Pareto frontier for the semi-batch MSJ model, valid for arbitrary stable arrival rates, numbers of servers, and service time distributions of large jobs. Through rigorous queueing-theoretic modeling and analysis, the study proves that the convoy policy family achieves Pareto optimality in the trade-off between the mean response time of large jobs and the throughput of small jobs.

Abstract

In large-scale computing systems, jobs often demand heterogeneous server allocations: large jobs that occupy a substantial fraction of the servers are of high importance and are thus latency-sensitive, while small jobs fill in the remaining capacity to maintain throughput. To model this dynamic, we introduce the half-batch multiserver-job (MSJ) framework, a queueing model in which large jobs arrive according to a Poisson process and require all servers simultaneously, while small jobs, each needing only one server, are always available. We prove that, in the half-batch MSJ model, the Pareto frontier for large-job mean response time and small-job throughput admits a simple and exact characterization. It is generated by a family of convoy policies, under which the system serves small jobs until $k$ large jobs have arrived and then switches to serving large jobs, together with convex combinations of neighboring convoy policies. Our result is fully general and non-asymptotic, holding for every stable arrival rate $λ$, every number of servers $n$, and every large-job size distribution $S$.

Problem

• Pareto-optimal
• multiserver-job
• scheduling
• response time
• throughput

Innovation

• Pareto-optimal scheduling
• multiserver-job model
• convoy policy
• queueing theory
• heterogeneous job allocation