From Good to Great: Improving Memory Tiering Performance Through Parameter Tuning

2025-04-25

AI Summary

Existing memory tiering systems rely on static thresholds and heuristic policies, rendering them ill-suited to diverse workloads and heterogeneous hardware—leading to suboptimal data placement and migration efficiency. To address this, we propose the first Bayesian optimization–based automated parameter-tuning framework specifically designed for memory tiering. Our method jointly models application runtime behavior and hardware-aware features to enable workload- and hardware-adaptive dynamic configuration of tiering parameters. It operates online atop mainstream tiering systems—including HeMem and HMSDK—to iteratively learn and optimize critical tiering thresholds. Experimental evaluation demonstrates that our approach achieves a 2.0× speedup over default configurations and outperforms the state-of-the-art tiering system by 1.56×. Moreover, it significantly improves cross-workload and cross-platform adaptability, establishing a new foundation for intelligent, self-tuning memory tiering.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Algorithm Configuration
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Memory tiering systems achieve memory scaling by adding multiple tiers of memory wherein different tiers have different access latencies and bandwidth. For maximum performance, frequently accessed (hot) data must be placed close to the host in faster tiers and infrequently accessed (cold) data can be placed in farther slower memory tiers. Existing tiering solutions employ heuristics and pre-configured thresholds to make data placement and migration decisions. Unfortunately, these systems fail to adapt to different workloads and the underlying hardware, so perform sub-optimally. In this paper, we improve performance of memory tiering by using application behavior knowledge to set various parameters (knobs) in existing tiering systems. To do so, we leverage Bayesian Optimization to discover the good performing configurations that capture the application behavior and the underlying hardware characteristics. We find that Bayesian Optimization is able to learn workload behaviors and set the parameter values that result in good performance. We evaluate this approach with existing tiering systems, HeMem and HMSDK. Our evaluation reveals that configuring the parameter values correctly can improve performance by 2x over the same systems with default configurations and 1.56x over state-of-the-art tiering system.

Problem

• Optimizing memory tiering performance through parameter tuning
• Adapting data placement to diverse workloads and hardware
• Enhancing performance using Bayesian Optimization for configuration

Innovation

• Leveraging Bayesian Optimization for parameter tuning
• Adapting to workload behaviors and hardware characteristics
• Improving performance through dynamic configuration adjustments