NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference

2026-07-16

AI Summary

This work addresses the limitations of conventional ReRAM-based in-memory computing (IMC) FPGA architectures, which struggle to efficiently handle the nonlinear and dynamic matrix operations prevalent in Transformers due to their reliance on static weights and high-overhead analog-to-digital converters (ADCs). To overcome these challenges, the authors propose a novel FPGA architecture featuring an ADC-free IMC module that leverages analog content-addressable memory (ACAM) to natively support nonlinear activation functions and dynamic matrix multiplication within memory. Coupled with an FPGA-aware design space exploration methodology, the approach optimizes crossbar dimensions and mapping strategies. The resulting architecture substantially broadens the applicability of IMC to complex models such as those employing attention mechanisms, achieving up to 40× and 1.9× improvements in energy efficiency, and 4.1× and 2.5× gains in area efficiency, on CNNs and Transformers, respectively.

Abstract

Recent FPGAs have improved deep learning (DL) inference efficiency through dedicated tensor blocks and in-BRAM computation. ReRAM-based analog in-memory computing (IMC) pushes efficiency further, offering an order-of-magnitude improvement in compute density and energy efficiency over conventional digital logic by performing vector-matrix multiplication (VMM) directly within the ReRAM crossbar; prior work has integrated such IMC blocks into FPGAs for DL inference. However, conventional IMC designs support only static-weight VMM, leaving nonlinear operations and dynamic matrix-matrix multiplication (DIMM) to the FPGA fabric. As a result, the benefits of IMC are largely confined to static-weight models, whereas Transformer-based models, which rely on frequent nonlinear and DIMM operations, gain only limited improvement. Moreover, the ADCs within each IMC block consume more than 70% of its area and power, further limiting system efficiency and scalability. To address these limitations, we propose a novel FPGA architecture that integrates an ADC-free IMC block, replacing the conventional ADC with analog content-addressable memories (ACAMs) that natively perform nonlinear operations inside the block. To fully exploit this block, we conduct an FPGA-aware design-space exploration that determines optimal crossbar dimensions while balancing FPGA area, flexibility, and DL performance, and we develop an efficient mapping that leverages ACAMs to carry out DIMM operations, extending the applicability of IMC to attention computation. On CNN and Transformer-based benchmarks, the proposed architecture achieves up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency, respectively. Overall, it significantly improves FPGA DL inference efficiency and sustains robust gains on Transformer-based workloads across long input sequences, advancing domain-specialized FPGA design.

Problem

• in-memory computing
• nonlinear operations
• dynamic matrix-matrix multiplication
• FPGA
• energy efficiency

Innovation

• ADC-free IMC
• analog content-addressable memory (ACAM)
• dynamic matrix-matrix multiplication (DIMM)
• FPGA-aware design-space exploration
• attention computation