RED: Energy Optimization Framework for eDRAM-based PIM with Reconfigurable Voltage Swing and Retention-aware Scheduling

2025-02-13

AI Summary

AI workloads such as Transformers suffer from high latency and energy consumption under the von Neumann architecture due to excessive data movement; while Processing-in-Memory (PIM) mitigates this, existing approaches overlook the dominant memory-access power in eDRAM macros. Method: We propose RED, an energy-optimized eDRAM-based PIM framework featuring: (i) runtime-reconfigurable read bitline (RBL) voltage swing; (ii) retention-time-aware refresh skipping; (iii) sense amplifier power gating; and (iv) a holistic optimization model jointly minimizing RBL swing, amplifier power, and refresh overhead, coupled with a multidimensional tiling-based energy-aware scheduling algorithm. Results: Experiments show RED reduces eDRAM macro energy by 74.88%, achieves 3.05× higher energy efficiency than SRAM-based PIM, and incurs only 3.5% area and 0.77% scheduling overhead.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Planning with Language Models
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

In the era of artificial intelligence (AI), Transformer demonstrates its performance across various applications. The excessive amount of parameters incurs high latency and energy overhead when processed in the von Neumann architecture. Processing-in-memory (PIM) has shown the potential in accelerating data-intensive applications by reducing data movement. While previous works mainly optimize the computational part of PIM to enhance energy efficiency, the importance of memory design, which consumes the most power in PIM, has been rather neglected. In this work, we present RED, an energy optimization framework for eDRAM-based PIM. We first analyze the PIM operations in eDRAM, obtaining two key observations: 1) memory access energy consumption is predominant in PIM, and 2) read bitline (RBL) voltage swing, sense amplifier power, and retention time are in trade-off relations. Leveraging them, we propose a novel reconfigurable eDRAM and retention-aware scheduling that minimizes the runtime energy consumption of the eDRAM macro. The framework pinpoints the optimal operating point by pre-estimating energy consumption across all possible tiling schemes and memory operations. Then, the reconfigurable eDRAM controls the RBL voltage swing at runtime according to the scheduling, optimizing the memory access power. Moreover, RED employs refresh skipping and sense amplifier power gating to mitigate the energy consumption overhead coming from the trade-off relation. Finally, the RED framework achieves up to 3.05x higher energy efficiency than the prior SRAM-based PIM, reducing the energy consumption of eDRAM macro up to 74.88% with reconfigurable eDRAM and optimization schemes, requiring only 3.5% area and 0.77% energy overhead for scheduling.

Problem

• Optimizes eDRAM-based PIM energy efficiency.
• Addresses high memory access power in PIM.
• Introduces reconfigurable voltage swing and scheduling.

Innovation

• Reconfigurable eDRAM for energy optimization
• Retention-aware scheduling minimizes runtime energy
• Refresh skipping and power gating reduce overhead