SLFS: a Flexible, Low-Cost Distributed File System Using Serverless Designs

2026-07-01

AI Summary

Traditional distributed file systems struggle to handle fine-grained, dynamically varying workloads due to coarse-grained resource reservation and scaling mechanisms, often resulting in resource underutilization and increased response latency. This work proposes the first fully serverless distributed file system, in which both data and metadata operations are delegated to short-lived, multithreaded functions. A policy-driven coordinator dynamically maps files to function instances, enabling fine-grained elasticity and cost efficiency. The design substantially mitigates cold-start overhead—reducing it by up to 580× compared to baseline approaches—and achieves comparable or superior performance while lowering costs by 63% relative to λFS, 68% relative to Amazon EFS, and 63% relative to Ceph.

Abstract

Large-scale distributed file systems must provision resources for peak demand, yet file access patterns fluctuate significantly, leaving substantial capacity idle during off-peak periods. Existing scaling mechanisms operate at the granularity of entire servers and take minutes to hours, making them unable to track the rapid, fine-grained load variations that file systems commonly experience. Serverless computing, with its millisecond-granularity elasticity and pay-per-use pricing, offers a compelling alternative. We present SLFS, the first distributed file system built with serverless functions for both data and metadata operations. SLFS implements file services on top of key-value stores, keeping function operations simple and short, and introduces a novel multi-threaded, short-lived server design that overcomes the cold-start problem while maintaining low cost. A policy-enforcing coordinator efficiently maps files to function instances, scales the system elastically, and controls function lifetimes to balance performance and cost. SLFS can flexibly run on diverse storage backends -- from cloud-native services like S3 to user-managed key-value stores -- enabling configurable cost-performance trade-offs. Our evaluation shows that SLFS mitigates cold starts by 580$\times$ compared to the base serverless design and outperforms $λ$FS, EFS, and Ceph at up to 63%, 68%, and 63% lower cost, respectively.

Problem

• distributed file system
• resource provisioning
• load fluctuation
• elastic scaling
• serverless computing

Innovation

• serverless file system
• cold-start mitigation
• elastic scaling
• key-value store
• cost-performance trade-off