Efficient Memory Tiering in a Virtual Machine

2025-06-06

AI Summary

In virtualized environments, host-level memory tiering schemes often misplace sparsely accessed large pages into expensive near-memory tiers due to memory access dispersion or skew, leading to resource inefficiency. To address this, we propose a lightweight, host-agnostic in-guest memory access aggregation mechanism that leverages two-level address translation to identify and reorganize fine-grained hot subpages—transforming sparsely hot large pages into densely hot ones. Our approach breaks from the conventional host-centric paradigm and is the first to enable runtime hotness-aware intra-page remapping in virtualized systems via dual-stage address translation. Experimental evaluation shows up to 50–70% reduction in near-memory utilization, 10–13% performance improvement under large-scale deployment, and no increase in memory total cost of ownership (TCO).

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search
• Multiagent Systems: Mechanism Design

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Memory tiering is the norm to effectively tackle the increasing server memory total cost of ownership (TCO) and the growing data demands of modern data center workloads. However, the host-based state-of-the-art memory tiering solutions can be inefficient for a virtualized environment when (i) the frequently accessed data are scattered across the guest physical address space or (ii) the accesses to a huge page inside the guest are skewed due to a small number of subpages being hot. Scattered or skewed accesses make the whole huge page look hot in the host address space. This results in host selecting and placing sparsely accessed huge pages in near memory, wasting costly near memory resources. We propose a host-agnostic technique employed inside the guest that exploits the two-level address translation in a virtualized environment to consolidate the scattered and skewed accesses to a set of guest physical address ranges. Consolidation transforms sparsely hot huge pages to densely hot huge pages in the host address space context. As a consequence, host-based tiering solutions can place densely hot huge pages in near memory, improving near memory utilization. Our evaluation of our technique on standalone real-world benchmarks with state-of-the-art host-based tiering show 50-70% reduction in near memory consumption at similar performance levels, while evaluation at scale improves performance by 10-13% with similar memory TCO.

Problem

• Reducing server memory costs in virtualized environments
• Optimizing memory tiering for scattered data access
• Improving near memory utilization for skewed workloads

Innovation

• Guest-based memory access consolidation technique
• Exploits two-level address translation
• Improves near memory utilization significantly