A TRRIP Down Memory Lane: Temperature-Based Re-Reference Interval Prediction For Instruction Caching

2025-09-17

AI Summary

Modern mobile CPUs suffer from high instruction reuse distances in complex applications, leading to frequent frontend stalls and diminished effectiveness of conventional cache replacement policies. To address this, this paper proposes TRRIP—a hardware-software co-design approach for instruction cache management. TRRIP uniquely integrates compiler-driven static “code temperature” analysis—based on profile-guided optimization (PGO)—with hardware-assisted Re-Reference Interval Prediction (RRIP) replacement. The compiler identifies hot code regions and annotates page-level temperature metadata, which is conveyed via a standardized OS interface to a lightweight hardware extension module that dynamically retains hot instructions in the instruction cache. Evaluated on optimized mobile applications, TRRIP reduces L2 instruction miss rates by 26.5% and delivers a geometric mean performance improvement of 3.9%, while maintaining bounded area and power overhead. This work significantly enhances the practicality and efficacy of instruction cache management in resource-constrained mobile environments.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Motion & Tracking

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Modern mobile CPU software pose challenges for conventional instruction cache replacement policies due to their complex runtime behavior causing high reuse distance between executions of the same instruction. Mobile code commonly suffers from large amounts of stalls in the CPU frontend and thus starvation of the rest of the CPU resources. Complexity of these applications and their code footprint are projected to grow at a rate faster than available on-chip memory due to power and area constraints, making conventional hardware-centric methods for managing instruction caches to be inadequate. We present a novel software-hardware co-design approach called TRRIP (Temperature-based Re-Reference Interval Prediction) that enables the compiler to analyze, classify, and transform code based on "temperature" (hot/cold), and to provide the hardware with a summary of code temperature information through a well-defined OS interface based on using code page attributes. TRRIP's lightweight hardware extension employs code temperature attributes to optimize the instruction cache replacement policy resulting in the eviction rate reduction of hot code. TRRIP is designed to be practical and adoptable in real mobile systems that have strict feature requirements on both the software and hardware components. TRRIP can reduce the L2 MPKI for instructions by 26.5% resulting in geomean speedup of 3.9%, on top of RRIP cache replacement running mobile code already optimized using PGO.

Problem

• Addressing instruction cache inefficiency in mobile CPUs
• Reducing hot code eviction through temperature-based prediction
• Optimizing cache replacement for complex mobile applications

Innovation

• Software-hardware co-design for cache optimization
• Compiler code temperature analysis and transformation
• Lightweight hardware extension for cache policy