Machine Learning-Driven Intelligent Memory System Design: From On-Chip Caches to Storage

2026-03-15

AI Summary

Traditional memory systems rely on static, handcrafted heuristics that struggle to adapt to dynamic workloads. This work presents the first systematic integration of lightweight machine learning techniques into the memory subsystem, introducing an adaptive, data-driven self-optimizing architecture. The proposed framework comprises Pythia, a reinforcement learning–based prefetcher; Hermes, a perceptron-driven off-chip bandwidth predictor; and Sibyl, a reinforcement learning–guided data placement policy. All three components consistently outperform state-of-the-art hand-tuned designs, delivering substantial improvements in both system performance and energy efficiency while incurring only modest hardware overhead.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Despite the data-rich environment in which memory systems of modern computing platforms operate, many state-of-the-art architectural policies employed in the memory system rely on static, human-designed heuristics that fail to truly adapt to the workload and system behavior via principled learning methodologies. In this article, we propose a fundamentally different design approach: using lightweight and practical machine learning (ML) methods to enable adaptive, data-driven control throughout the memory hierarchy. We present three ML-guided architectural policies: (1) Pythia, a reinforcement learning-based data prefetcher for on-chip caches, (2) Hermes, a perceptron learning-based off-chip predictor for multi-level cache hierarchies, and (3) Sibyl, a reinforcement learning-based data placement policy for hybrid storage systems. Our evaluation shows that Pythia, Hermes, and Sibyl significantly outperform the best-prior human-designed policies, while incurring modest hardware overheads. Collectively, this article demonstrates that integrating adaptive learning into memory subsystems can lead to intelligent, self-optimizing architectures that unlock performance and efficiency gains beyond what is possible with traditional human-designed approaches.

Problem

• memory system
• adaptive learning
• machine learning
• workload adaptation
• intelligent architecture

Innovation

• machine learning
• memory hierarchy
• reinforcement learning
• adaptive prefetching
• hybrid storage