Opportunistic Scheduling for Optimal Spot Instance Savings in the Cloud
This study addresses the problem of scheduling delay-sensitive tasks to spot and on-demand cloud instances under an average latency constraint, aiming to minimize average cost. By modeling the system using queueing theory and stochastic processes, and leveraging convex optimization and knapsack problem analysis, the work characterizes the optimal scheduling structure in both low- and high-latency regimes: it proves that a queue length of one is optimal in the former, while in the latter, it designs an approximation-optimal policy based on knapsack formulation. An adaptive scheduling algorithm is further proposed to dynamically exploit the allowable latency window. Experimental results demonstrate that the algorithm achieves near-theoretical-optimal cost while effectively balancing latency constraints and resource expenditure. This work provides the first analytical solution for scheduling across hybrid spot and on-demand instances under latency guarantees.