Architectural and System Implications of CXL-enabled Tiered Memory

2025-03-22

AI Summary

This work addresses system-level challenges introduced by CXL memory—including high latency, limited parallelism, and severe interference with DDR bandwidth (up to 81%)—tracing their root causes to unfair request scheduling, degraded inter-core synchronization, and contention for shared hardware resources. To mitigate these issues, we propose MIKU, a dynamic memory request regulation mechanism. MIKU introduces the first service-time-aware, cooperative priority scheduler for CXL and DDR requests, enabling fair allocation and efficiency-oriented coordination across heterogeneous memory resources. Leveraging microbenchmark-driven analysis, cycle-accurate hardware modeling, and protocol-stack co-optimization, MIKU recovers 98% of peak DDR bandwidth under heavy load, reduces CXL average latency variation by 42%, and significantly improves overall memory efficiency.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Memory disaggregation is an emerging technology that decouples memory from traditional memory buses, enabling independent scaling of compute and memory. Compute Express Link (CXL), an open-standard interconnect technology, facilitates memory disaggregation by allowing processors to access remote memory through the PCIe bus while preserving the shared-memory programming model. This innovation creates a tiered memory architecture combining local DDR and remote CXL memory with distinct performance characteristics. In this paper, we investigate the architectural implications of CXL memory, focusing on its increased latency and performance heterogeneity, which can undermine the efficiency of existing processor designs optimized for (relatively) uniform memory latency. Using carefully designed micro-benchmarks, we identify bottlenecks such as limited hardware-level parallelism in CXL memory, unfair queuing in memory request handling, and its impact on DDR memory performance and inter-core synchronization. Our findings reveal that the disparity in memory tier parallelism can reduce DDR memory bandwidth by up to 81% under heavy loads. To address these challenges, we propose a Dynamic Memory Request Control mechanism, MIKU, that prioritizes DDR memory requests while serving CXL memory requests on a best-effort basis. By dynamically adjusting CXL request rates based on service time estimates, MIKU achieves near-peak DDR throughput while maintaining high performance for CXL memory. Our evaluation with micro-benchmarks and representative workloads demonstrates the potential of MIKU to enhance tiered memory system efficiency.

Problem

• Investigates CXL memory's latency and performance heterogeneity impact
• Identifies bottlenecks in CXL memory parallelism and queuing
• Proposes MIKU mechanism to optimize tiered memory efficiency

Innovation

• CXL enables tiered memory via PCIe interconnect
• Dynamic Memory Request Control prioritizes DDR access
• MIKU adjusts CXL requests for optimal performance