GraphFlash: Enabling Fast and Elastic Graph Processing on Serverless Infrastructure

2026-05-12

AI Summary

This work addresses the poor elasticity and low resource efficiency of existing graph processing systems under dynamic workloads, as well as the limitations of current serverless approaches—namely, inefficient state management and high communication overhead. To overcome these challenges, we propose GraphFlash, a serverless-based graph processing framework that innovatively integrates a subgraph-centric programming model with shared external storage to enable stateless, fine-grained function execution. GraphFlash introduces several key optimizations, including partition-aware key aggregation, intra-function partition colocation, and superstep-aware activation, and supports both rotating and fixed execution modes to adapt to varying resource conditions. Experimental results demonstrate that GraphFlash achieves up to 127× speedup and 98% lower resource consumption across multiple graph algorithms; even under severe resource constraints, it delivers 48× acceleration and 99.97% cost reduction while matching the performance of conventional distributed frameworks.

Abstract

Graph processing systems are essential for analyzing large-scale data with complex relationships, yet most existing frameworks rely on statically provisioned clusters, resulting in poor elasticity and inefficient resource utilization under dynamic workloads. Serverless computing offers automatic scaling and fine-grained billing, but existing serverless graph systems suffer from performance limitations due to inefficient state management and high communication overhead through external storage. We present GraphFlash, a fast and elastic graph processing framework built on serverless infrastructure. GraphFlash adopts a subgraph-centric programming model and leverages shared external storage for coordination and communication, enabling stateless, fine-grained function execution. It supports two execution modes: rotating mode for resource-constrained environments and pinned mode for higher performance when resources are sufficient. To address serverless limitations, GraphFlash introduces system-level optimizations, including partition-aware key aggregation, intra-function partition co-location, and superstep-aware activation. Across multiple graph algorithms and datasets, GraphFlash outperforms existing serverless-compatible systems by up to 127x in execution time and reduces resource consumption by up to 98% under higher-resource configurations, while matching the performance of traditional distributed frameworks on large workloads. Even with limited resources, it achieves up to 48x speedup and 99.97% cost reduction over prior serverless solutions, demonstrating that GraphFlash makes serverless graph processing practical and performant.

Problem

• graph processing
• serverless computing
• elasticity
• resource utilization
• communication overhead

Innovation

• serverless graph processing
• subgraph-centric model
• partition-aware aggregation
• elastic computing
• stateless execution