Idiosyncrasies of Programmable Caching Engines

2026-03-15

AI Summary

In dynamic multi-tenant environments, programmable caching engines such as CacheLib often suffer from performance degradation, memory inefficiency, and unfair service allocation due to rigid configuration schemes, insufficient runtime adaptability, and the absence of quality-of-service (QoS) guarantees. This work presents the first systematic empirical evaluation of CacheLib under fluctuating workloads across a range of configurations, uncovering its critical bottlenecks and limitations. The study not only quantifies the shortcomings of current designs in terms of fairness and efficiency but also provides clear guidance for future enhancements aimed at improving dynamic adaptability, QoS support, and programmability in caching systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Programmable caching engines like CacheLib are widely used in production systems to support diverse workloads in multi-tenant environments. CacheLib's design focuses on performance, portability, and configurability, allowing applications to inherit caching improvements with minimal implementation effort. However, its behavior under dynamic and evolving workloads remains largely unexplored. This paper presents an empirical study of CacheLib with multi-tenant settings under dynamic and volatile environments. Our evaluation across multiple CacheLib configurations reveals several limitations that hinder its effectiveness under such environments, including rigid configurations, limited runtime adaptability, lack of quality-of-service support and coordination, which lead to suboptimal performance, inefficient memory usage, and tenant starvation. Based on these findings, we outline future research directions to improve the adaptability, fairness, and programmability of future caching engines.

Problem

• programmable caching
• multi-tenant
• dynamic workloads
• quality-of-service
• cache adaptability

Innovation

• programmable caching
• multi-tenant caching
• dynamic workloads
• cache adaptability
• quality of service