SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost

2026-01-10

arXiv.org

AI Summary

This work addresses the challenge of executing AI batch jobs within strict deadlines while minimizing cost on volatile cloud spot instances, whose unpredictable interruptions complicate the trade-off between affordability and reliability. The authors propose a novel multi-region scheduling approach that systematically exploits the spatiotemporal heterogeneity of spot instances—across price, availability, and lifetime—by integrating lightweight probing, lifetime prediction, and migration cost modeling to dynamically select optimal instances. Evaluated in real-world cloud environments, the method achieves 1.25–3.96× cost savings while consistently meeting job deadlines. Simulations further demonstrate that its incurred costs remain within 10% of the theoretical optimum, substantially outperforming existing strategies.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Hardware-aware ML

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-10x lower cost than on-demand instances, but their unpredictable availability makes meeting deadlines difficult. Existing systems either rely solely on spot instances and risk deadline violations, or operate in simplified single-region settings. These approaches overlook substantial spatial and temporal heterogeneity in spot availability, lifetimes, and prices. We show that exploiting such heterogeneity to access more spot capacity is the key to reduce the job execution cost. We present SkyNomad, a multi-region scheduling system that maximizes spot usage and minimizes cost while guaranteeing deadlines. SkyNomad uses lightweight probing to estimate availability, predicts spot lifetimes, accounts for migration cost, and unifies regional characteristics and deadline pressure into a monetary cost model that guides scheduling decisions. Our evaluation shows that SkyNomad achieves 1.25-3.96x cost savings in real cloud deployments and performs within 10% cost differences of an optimal policy in simulation, while consistently meeting deadlines.

Problem

• spot instances
• AI batch jobs
• deadline constraints
• multi-region scheduling
• cost minimization

Innovation

• multi-region scheduling
• spot instances
• cost optimization
• deadline guarantee
• heterogeneity exploitation