Hiku: Pull-Based Scheduling for Serverless Computing

2025-02-21

AI Summary

To address scheduling bottlenecks in serverless computing—including load imbalance, frequent cold starts, and poor adaptability to bursty traffic—this paper proposes the first pull-based scheduling mechanism for Serverless platforms: idle worker nodes proactively pull tasks, decoupling task dispatching from node selection and enabling a worker-driven task acquisition paradigm. We implement a lightweight scheduler atop an open-source FaaS platform, integrating modules for idle-node detection, queue-based task pulling, and load-aware task distribution. Experimental evaluation demonstrates that, compared to conventional hash-based scheduling, our approach reduces average response latency by 14.9%, decreases cold-start rate from 43% to 30%, improves throughput by 8.3%, and enhances load balance by 12.9%. These gains collectively yield significantly improved responsiveness and elasticity under dynamic workloads.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search

Application Category

• Economics, Online Markets and Human Computation: The sharing economy
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Serverless computing promises convenient abstractions for developing and deploying functions that execute in response to events. In such Function-as-a-Service (FaaS) platforms, scheduling is an integral task, but current scheduling algorithms often struggle with maintaining balanced loads, minimizing cold starts, and adapting to commonly occurring bursty workloads. In this work, we propose pull-based scheduling as a novel scheduling algorithm for serverless computing. Our key idea is to decouple worker selection from task assignment, with idle workers requesting new tasks proactively. Experimental evaluation on an open-source FaaS platform shows that pull-based scheduling, compared to other existing scheduling algorithms, significantly improves the performance and load balancing of serverless workloads, especially under high concurrency. The proposed algorithm improves response latencies by 14.9% compared to hash-based scheduling, reduces the frequency of cold starts from 43% to 30%, increases throughput by 8.3%, and achieves a more even load distribution by 12.9% measured by the requests assigned per worker.

Problem

• Improves load balancing in serverless computing
• Reduces cold start frequency in FaaS platforms
• Enhances performance under high concurrency workloads

Innovation

• Pull-based scheduling algorithm
• Decouples worker selection
• Improves load balancing performance