Jenga: Responsive Tiered Memory Management without Thrashing

2025-10-26

AI Summary

In heterogeneous memory systems, page thrashing—caused by mixing cold and hot data within the same page and sudden shifts in data access热度—severely degrades performance. This paper proposes a hierarchical memory management mechanism tailored for CXL and non-volatile memory (NVM). First, it introduces a context-aware page allocation policy to prevent colocation of cold and hot data. Second, it designs a low-overhead, high-accuracy dynamic hotness estimation model that detects access pattern shifts within milliseconds. Third, it integrates hotness-aware scheduling with fine-grained, on-demand page migration to minimize migration overhead. Evaluated on ten memory-intensive workloads, the approach achieves an average 28% performance improvement, incurs less than 3% single-core CPU overhead, and consumes under 0.3% memory for metadata—significantly outperforming state-of-the-art solutions.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

A heterogeneous memory has a single address space with fast access to some addresses (a fast tier of DRAM) and slow access to other addresses (a capacity tier of CXL-attached memory or NVM). A tiered memory system aims to maximize the number of accesses to the fast tier via page migrations between the fast and capacity tiers. Unfortunately, previous tiered memory systems can perform poorly due to (1) allocating hot and cold objects in the same page and (2) abrupt changes in hotness measurements that lead to thrashing. This paper presents Jenga, a tiered memory system that addresses both problems. Jenga's memory allocator uses a novel context-based page allocation strategy. Jenga's accurate measurements of page hotness enable it to react to memory access behavior changes in a timely manner while avoiding thrashing. Compared to the best previous tiered memory system, Jenga runs memory-intensive applications 28% faster across 10 applications, when the fast tier capacity matches the working set size, at a CPU overhead of <3% of a single core and a memory overhead of <0.3%

Problem

• Prevents performance degradation from hot-cold object allocation
• Eliminates thrashing caused by abrupt hotness measurement changes
• Optimizes page migration between fast and slow memory tiers

Innovation

• Context-based page allocation for hot/cold separation
• Accurate page hotness measurement to avoid thrashing
• Responsive tiered memory management with low overhead