Melding the Serverless Control Plane with the Conventional Cluster Manager for Speed and Compatibility

2025-05-30

AI Summary

Existing Kubernetes-based control planes struggle to simultaneously minimize cold-start latency and maximize resource utilization under highly volatile, tightly coupled serverless workloads, leading to widespread instance idleness and CPU/memory waste. Method: We propose a dual-track control plane: a primary track supporting asynchronous scheduling for functional completeness, and a secondary local Wavelet fast path that bypasses the cluster manager to enable lightweight emergency instances within seconds. Key innovations include a node-local Wavelet proxy, a dual-mode scheduling mechanism, an ultra-lightweight emergency instance runtime, and a hybrid control-flow integration architecture. Results: Our system handles over 98% of requests via conventional cluster managers while achieving 35% lower end-to-end latency than Dirigent at comparable cost; against synchronous and asynchronous Kubernetes baselines, it reduces latency by 1.5–3.5× and cuts costs by 8–33%.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Hardware-aware ML
• Multiagent Systems: Coordination and Collaboration

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: The sharing economy

Abstract

Modern serverless applications, often interactive with highly volatile traffic, challenge system scalability, demanding control planes that deliver low latency and cost efficiency. Analysis of production traces and existing systems reveals that current control plane designs (synchronous and asynchronous), particularly when built on conventional cluster managers like Kubernetes, struggle with this balance, often wasting significant CPU and memory resources on creating underutilized or idle instances. While clean-slate approaches like Dirigent offer performance gains, they sacrifice compatibility with established cluster management ecosystems. We introduce WaveLink, a serverless system designed to achieve high performance and low cost while maintaining compatibility with conventional cluster managers. WaveLink employs a novel dual-track control plane. A standard asynchronous track manages long-lived, full-featured regular instances for handling predictable, sustainable traffic, preserving full compatibility and feature sets off the critical path. Concurrently, an expedited parallel track addresses excessive traffic bursts that trigger cold starts. This fast path utilizes node-local agents (Wavelets) to rapidly spawn short-lived Emergency Instances with a reduced feature set, critically bypassing the latency overhead of the main cluster manager. Our experiments demonstrate that WaveLink, while remaining compatible with conventional managers for>98% invocation traffic, achieves 35% faster end-to-end performance at a comparable cost to the incompatible Dirigent system. WaveLink outperforms Kubernetes-compatible systems with synchronous control planes by 1.5-3.5x at 8-21% lower cost, and surpasses asynchronous counterparts by 1.7-3.5x at 3-33% lower cost.

Problem

• Balancing low latency and cost efficiency in serverless control planes
• Addressing resource wastage in conventional cluster managers like Kubernetes
• Maintaining compatibility while improving performance in serverless systems

Innovation

• Dual-track control plane for performance and compatibility
• Node-local agents for rapid cold start handling
• Emergency Instances with reduced feature set