Heterogeneous Memory Design Exploration for AI Accelerators with a Gain Cell Memory Compiler

2026-02-24

AI Summary

On-chip memory area and energy consumption have become critical bottlenecks limiting the performance of AI accelerators. To address this challenge, this work introduces high-density, low-power Gain Cell RAM (GCRAM) into the AI accelerator memory design space for the first time and presents OpenGCRAM, an open-source compiler that enables automatic generation, placement, and retention-time-tunable configuration of both SRAM and GCRAM macros. Integrated with commercial CMOS process libraries and coupled with a configuration search methodology tailored to AI workloads, the compiler facilitates a systematic exploration of heterogeneous memory architectures. Experimental results quantitatively demonstrate the advantages of various SRAM/GCRAM hybrid configurations in terms of area, latency, and power, offering an efficient and customizable heterogeneous memory solution for AI accelerators.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Search and Optimization: Algorithm Configuration

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

As memory increasingly dominates system cost and energy, heterogeneous on-chip memory systems that combine technologies with complementary characteristics are becoming essential. Gain Cell RAM (GCRAM) offers higher density, lower power, and tunable retention, expanding the design space beyond conventional SRAM. To this end, we create an OpenGCRAM compiler supporting both SRAM and GCRAM. It generates macro-level designs and layouts for commercial CMOS processes and characterizes area, delay, and power across user-defined configurations. The tool enables systematic identification of optimal heterogeneous memory configurations for AI tasks under specified performance metrics.

Problem

• Heterogeneous Memory
• AI Accelerators
• Gain Cell RAM
• Memory Design Exploration
• On-chip Memory

Innovation

• Heterogeneous Memory
• Gain Cell RAM (GCRAM)
• Memory Compiler
• AI Accelerator
• On-chip Memory