A Joint Learning Approach to Hardware Caching and Prefetching

2025-10-12

AI Summary

This work identifies a bidirectional dependency between cache replacement and prefetching policies—a relationship that degrades co-design performance when addressed via independent training. To address this, we propose the first joint learning framework: a shared feature encoder models their mutual interdependence, while contrastive learning enhances representation learning of memory access patterns. Our approach is the first to theoretically establish and empirically exploit this bidirectional coupling, enabling end-to-end co-optimization under dynamic workloads. Experimental evaluation across mainstream benchmarks demonstrates that our method achieves average improvements of 3.2% in cache hit rate and 5.7% in prefetch accuracy over independently trained baselines. These results substantiate a novel paradigm for intelligent, synergistic cache optimization—advancing beyond isolated policy design toward holistic, data-driven co-adaptation of replacement and prefetching.

Technology Category

• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Several learned policies have been proposed to replace heuristics for scheduling, caching, and other system components in modern systems. By leveraging diverse features, learning from historical trends, and predicting future behaviors, such models promise to keep pace with ever-increasing workload dynamism and continuous hardware evolution. However, policies trained in isolation may still achieve suboptimal performance when placed together. In this paper, we inspect one such instance in the domain of hardware caching -- for the policies of cache replacement and prefetching. We argue that these two policies are bidirectionally interdependent and make the case for training the two jointly. We propose a joint learning approach based on developing shared representations for the features used by the two policies. We present two approaches to develop these shared representations, one based on a joint encoder and another based on contrastive learning of the embeddings, and demonstrate promising preliminary results for both of these. Finally, we lay down an agenda for future research in this direction.

Problem

• Optimizing interdependent hardware caching and prefetching policies
• Addressing suboptimal performance from isolated policy training
• Developing shared feature representations for joint learning

Innovation

• Joint learning approach for hardware caching
• Shared representations between caching policies
• Contrastive learning and joint encoder methods