Taming Serverless Cold Starts Through OS Co-Design

2025-09-16

AI Summary

To address high cold-start latency in serverless computing—where functions incur significant initialization overhead upon first invocation after prolonged idleness—this paper introduces Spice, a novel execution engine featuring the first OS-level co-design of snapshotting and restoration. Unlike conventional approaches relying on memory residency or kernel-state replay, Spice integrates a kernel-resident, purpose-built memory-mapping restoration primitive to efficiently load disk-based snapshots, thereby minimizing page faults and I/O overhead. Its key innovation lies in bypassing kernel-state reconstruction entirely and directly restoring user-space memory layouts, enabling cold starts that closely match warm-start performance. Evaluation demonstrates that Spice reduces cold-start latency by 14.9× and 10.6× compared to process- and VM-based systems, respectively, achieving sub-millisecond restoration while preserving memory elasticity and reliability.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Data transparency and provenance

Abstract

Serverless computing promises fine-grained elasticity and operational simplicity, fueling widespread interest from both industry and academia. Yet this promise is undercut by the cold setart problem, where invoking a function after a period of inactivity triggers costly initialization before any work can begin. Even with today's high-speed storage, the prevailing view is that achieving sub-millisecond cold starts requires keeping state resident in memory. This paper challenges that assumption. Our analysis of existing snapshot/restore mechanisms show that OS-level limitations, not storage speed, are the real barrier to ultra-fast restores from disk. These limitations force current systems to either restore state piecemeal in a costly manner or capture too much state, leading to longer restore times and unpredictable performance. Futhermore, current memory primitives exposed by the OS make it difficult to reliably fetch data into memory and avoid costly runtime page faults. To overcome these barriers, we present Spice, an execution engine purpose-built for serverless snapshot/restore. Spice integrates directly with the OS to restore kernel state without costly replay and introduces dedicated primitives for restoring memory mappings efficiently and reliably. As a result, Spice delivers near-warm performance on cold restores from disk, reducing latency by up to 14.9x over state-of-the-art process-based systems and 10.6x over VM-based systems. This proves that high performance and memory elasticity no longer need to be a trade-off in serverless computing.

Problem

• Addressing serverless cold start latency from disk restores
• Overcoming OS limitations for efficient snapshot and restore
• Reducing initialization delays without keeping state in memory

Innovation

• OS co-design for serverless cold starts
• Direct OS integration to restore kernel state
• Dedicated primitives for efficient memory mapping