prefetching strategy design

Memory latency and bandwidth bottlenecks continue to impede system performance in the post-Moore era. To address this, we present the first systematic demonstration that software prefetching—under near-data processing (NDP) architectures—exhibits superior scalability and efficiency over conventional hardware prefetching. We propose a lightweight preloading paradigm tailored for intelligent memory, which jointly optimizes latency hiding and computational resource utilization via software-driven interleaving of computation and I/O scheduling, coupled with memory-bandwidth-aware load preloading. Experimental evaluation shows that our approach significantly improves compute-unit utilization, and its prefetching efficiency consistently increases with CPU process node advancements. Across multiple generations of hardware platforms, it achieves superior latency-hiding effectiveness compared to state-of-the-art hardware prefetchers.

This work addresses the problem of excessive and ineffective hardware prefetching in datacenter workloads, which wastes precious memory bandwidth. The authors propose a novel hardware-software cooperative prefetching mechanism that leverages page table entries to convey page-level prefetch hints, enabling dynamic control over hardware prefetcher behavior without requiring modifications to the instruction set architecture or application binaries. The approach is compatible with existing state-of-the-art prefetchers—such as BOP, SPP+PPF, and Pythia—and selectively disables prefetching on non-critical pages at runtime to balance coverage and bandwidth efficiency. Experimental evaluation demonstrates that the proposed technique reduces ineffective prefetch requests by approximately 40% under representative datacenter workloads and achieves performance improvements of up to 4.1%.

Existing L1 instruction prefetchers struggle to operate efficiently due to address translation delays caused by page boundary crossings and the highly heterogeneous reuse behavior of prefetched cache lines. To address these limitations, this work proposes IP-CaT, a novel framework that jointly optimizes TLB and cache management for improved instruction prefetching. IP-CaT introduces a translation prefetch buffer (tPB) to reduce translation overhead and incorporates a tri-mode instruction prefetch replacement policy (TIPRP) tailored to diverse reuse patterns. Evaluated across 105 server workloads, IP-CaT combined with EPI achieves an average performance improvement of 8.7%, significantly outperforming state-of-the-art TLB prefetching mechanisms as well as advanced cache management strategies such as CHiRP, Emissary, SHiP++, and Mockingjay.

Hardware prefetchers sharing limited resources often suffer from request conflicts, while existing selection mechanisms exhibit two critical deficiencies: inaccurate demand allocation and coarse-grained decision making. This paper proposes a synergistic mechanism combining dynamic demand-aware request allocation with fine-grained prefetcher selection. We introduce customized request routing—novel for prefetching—to enable timing-sensitive prefetch table management and low-overhead pattern matching. Furthermore, we design Alecto, a lightweight, rule-based algorithm incorporating adaptive decision logic to support real-time request分流. Evaluation shows that our approach improves single-core and 8-core performance by 2.76% and 7.56%, respectively, over the RL-based Bandit baseline; achieves 5.25% average speedup on memory-intensive benchmarks; reduces prefetch table access energy by 48%; and incurs less than 1 KB of storage overhead.

To address the low last-level cache (LLC) prefetching efficiency caused by irregular memory access patterns, this paper proposes a programmable and scalable software-defined LLC prefetcher. Unlike conventional hardware-based prediction mechanisms, our design fully migrates prefetching logic to the software layer, enabling lightweight API-driven specification of access patterns without requiring instruction-set extensions—thereby significantly reducing hardware complexity and improving resource utilization. We implement the prefetcher in gem5 as a hybrid architecture that cooperates with private-cache prefetchers. Experimental evaluation on the GAPBS graph traversal benchmark shows up to 1.74× speedup over the baseline system; when jointly deployed with private-cache prefetchers, it still achieves 1.40× speedup. These results demonstrate the prefetcher’s effectiveness in adapting to complex, irregular memory access patterns.

Existing approaches struggle to effectively coordinate data prefetching with off-chip prediction, and their control strategies often suffer from low efficiency. This work formulates the joint optimization of these two mechanisms as an online reinforcement learning problem for the first time, introducing a unified framework that enables end-to-end adaptive scheduling. By dynamically adjusting the activation policies and aggressiveness levels of individual components based on system-level state monitoring—such as accuracy and bandwidth utilization—the proposed method achieves coordinated control in real time. Experimental results demonstrate that the approach significantly outperforms state-of-the-art solutions across a range of memory-intensive workloads and system configurations, delivering substantial performance gains while maintaining low storage overhead.

This work addresses the challenge of irregular memory accesses, which lack temporal or spatial address patterns and thus evade efficient handling by conventional prefetchers, often at high storage cost. The paper proposes Instruction Correlation-based Prefetching (ICP), a novel mechanism that abandons assumptions about address correlation and instead exploits stable data dependencies among instructions that generate irregular memory accesses. By leveraging outcomes of already-executed instructions to predict future access addresses, ICP achieves lightweight yet effective prefetching. Requiring only 2.1 KB of storage—three orders of magnitude smaller than comparable approaches—it outperforms state-of-the-art temporal prefetchers, delivering speedups of 14.0% over Triangel and 6.0% over DMP on SPEC CPU and GAP benchmarks, respectively.

This work addresses the limitations of traditional hardware prefetchers, which rely on manual analysis of program traces and struggle to handle performance anomalies in real-world workloads. The authors propose a performance-anomaly-driven automated design methodology that employs an intelligent agent to diagnose prefetching failures, automatically synthesize a Mixture-of-Prefetchers (MoP), and continuously refine it through simulation. This approach represents the first practical, agent-driven framework capable of generating synthesizable RTL for prefetchers, outperforming state-of-the-art handcrafted designs on previously unseen workloads. Evaluated on SPEC CPU2006/2017, MoP achieves a 61.1% geometric mean IPC improvement over no prefetching and surpasses Alecto, Berti, and Pythia by 14.5%–23.6%. Its RTL implementation in a 6nm process occupies only 110KB of storage and 0.0347mm² of silicon area.

This work addresses the growing challenge posed by the expanding data footprint of modern applications, which renders memory systems a critical bottleneck for both performance and energy efficiency—constraints that traditional microarchitectures struggle to overcome. To this end, the paper introduces a data-driven microarchitectural design paradigm that systematically integrates lightweight machine learning with application-specific data semantic features across multiple processor components. Key contributions include a reinforcement learning–based hardware prefetcher, a perceptron-driven off-chip access predictor, a synergistic mechanism coordinating prefetching and prediction, and a predictability-aware memory access elimination technique leveraging both address and value repetition. Experimental results demonstrate that the proposed approach substantially outperforms state-of-the-art solutions, delivering significant improvements in both performance and energy efficiency.

This work proposes ORAP, a novel prefetcher that jointly optimizes prefetching strategies and Rowhammer mitigation for the first time. By caching DRAM row buffer contents in the last-level cache (LLC), ORAP significantly reduces redundant row activations. It integrates LLC management, the Berti prefetcher, and DDR5-native Rowhammer countermeasures to enable row-activation-aware prefetching. Experimental results demonstrate that, under Refresh Management (RFM), ORAP reduces DRAM activation rates by 51.3% and improves performance by 4.6% compared to Berti+SPP-PPF. Under Probabilistic Row Activation Counting (PRAC), ORAP further reduces energy overhead by 11.8%. These gains highlight ORAP’s effectiveness in reconciling the tension between aggressive prefetching and Rowhammer protection.