Making Serverless Computing Extensible: A Case Study of Serverless Data Analytics

2025-07-16

AI Summary

Serverless computing struggles to simultaneously achieve scalability, generality, and usability under complex workloads. Method: This paper introduces the “decision workflow” abstraction, modeling control-plane behavior as programmable, composable high-level decision sequences—enabling developers to customize domain-specific optimizations on shared platforms. Based on this abstraction, we design a programmable control plane and fine-grained resource-sharing policies, implementing Proteus, a prototype system tailored for data analytics. Results: Experiments show that Proteus improves analytical query execution efficiency by up to 4.2×, while enabling dynamic resource reuse across heterogeneous applications. Crucially, it achieves this without sacrificing serverless generality or developer simplicity—marking the first solution that unifies control-plane extensibility with domain-specific performance optimization.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Distributed Search
• Machine Learning: Scalability of ML Systems

Application Category

• Economics, Online Markets and Human Computation: The sharing economy
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Serverless computing has attracted a broad range of applications due to its ease of use and resource elasticity. However, developing serverless applications often poses a dilemma -- relying on general-purpose serverless platforms can fall short of delivering satisfactory performance for complex workloads, whereas building application-specific serverless systems undermines the simplicity and generality. In this paper, we propose an extensible design principle for serverless computing. We argue that a platform should enable developers to extend system behaviors for domain-specialized optimizations while retaining a shared, easy-to-use serverless environment. We take data analytics as a representative serverless use case and realize this design principle in Proteus. Proteus introduces a novel abstraction of decision workflows, allowing developers to customize control-plane behaviors for improved application performance. Preliminary results show that Proteus's prototype effectively optimizes analytical query execution and supports fine-grained resource sharing across diverse applications.

Problem

• Extending serverless platforms for domain-specific optimizations
• Balancing performance and simplicity in serverless computing
• Enhancing data analytics performance in shared serverless environments

Innovation

• Extensible design principle for serverless computing
• Decision workflows abstraction for customization
• Optimizes query execution and resource sharing