WarmSwap: Sharing Dependencies for Accelerating Cold Starts in Serverless Functions

2024-09-13

arXiv.org

AI Summary

To address the prolonged dependency loading latency during Serverless cold starts—and the inability of existing optimizations to simultaneously improve tail-latency function performance and comply with cloud providers’ caching constraints—this paper proposes a provider-side dependency sharing optimization mechanism. Its core innovation is the first “dependency-category–aware” intermediate-state optimization paradigm: pre-initialized dependency images are migrated at container startup, enabling multiple functions to share a single image and thereby transcending the binary limitations of infrastructure-level or function-level optimizations. Key techniques include dependency image migration, cache-aware image-sharing scheduling, and runtime hot-swap of container dependencies. Evaluation on seven FunctionBench workloads demonstrates 2.2–3.2× speedup in dependency loading; trace-driven simulation based on Azure production traces shows that sharing one dependency image among ten functions reduces optimization overhead by 88%.

Technology Category

• Search and Optimization: Distributed Search
• Machine Learning: Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Cloud, edge and content delivery systems for the Web
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This work presents WarmSwap, a novel provider-side cold-start optimization for serverless computing. This optimization reduces cold-start time when booting and loading dependencies at runtime inside a function container. Previous approaches to the optimization of cold starts tend to fall into two categories: optimizing the infrastructure of serverless computing to benefit all serverless functions; or function-specific tuning for individual serverless functions. In contrast, WarmSwap offers a broad middle ground, which optimizes entire categories of serverless functions. WarmSwap eliminates the need to initialize middleware or software dependencies when launching a new serverless container, by migrating a pre-initialized live dependency image to the new function instance. WarmSwap respects the provider's cache constraints, as a single pre-warmed dependency image in the cache is shared among all serverless functions requiring that software dependency image. WarmSwap has been tested on seven representative functions from FunctionBench. In those tests, WarmSwap accelerates dependency loading for serverless functions with large dependency requirements by a factor ranging from 2.2 to 3.2. Simulation experiments using Azure traces indicate that WarmSwap can save 88% of optimization space when sharing a dependency image among ten different functions.

Problem

• Reducing cold-start latency in serverless computing
• Optimizing cross-function dependency sharing workloads
• Respecting cache constraints while pre-warming dependencies

Innovation

• Pre-warming strategy for live dependency sharing
• Cross-function optimization with shared dependency images
• Respects cache constraints while reducing cold-start