XtraMAC: An Efficient MAC Architecture for Mixed-Precision LLM Inference on FPGA

2026-05-07

AI Summary

Existing FPGA-based MAC architectures struggle to efficiently support the mixed-precision quantization required by large language models, suffering from low resource utilization and limited parallelism. This work proposes an adaptive MAC microarchitecture that, for the first time, unifies integer, floating-point, and mixed-precision operations within a single hardware unit. By decomposing operands into shared integer mantissa products and augmenting them with lightweight sign and exponent handling, the design enables dynamic operand packing and highly efficient DSP reuse. Implemented on the AMD Xilinx U55c platform, the architecture achieves constant latency and initiation interval, delivering 1.4–2.0× higher computational density, 27–51% reductions in LUT, FF, and DSP usage, up to 1.9× improved energy efficiency, and a 1.2× speedup in mixed-precision LLM inference compared to state-of-the-art solutions.

Abstract

The widespread adoption of mixed-precision quantization in large language models (LLMs) has created demand for hardware that can efficiently perform multiply-accumulate (MAC) operations across mixed datatypes and switch datatypes at runtime. Existing FPGA-based MAC solutions fall short due to limitations in fixed-datatype design, inefficient spatial or temporal resource sharing, and poor support for mixed-precision execution. These limitations collectively lead to under-utilization of DSP resources, limiting achievable parallelism and throughput. In this work, we present XtraMAC, a novel MAC architecture that unifies integer, floating-point, and mixed-precision operations within a single, datatype-adaptive microarchitecture. XtraMAC decomposes all supported MAC formats into a shared integer mantissa product with lightweight sign and exponent handling, enabling dynamic operand packing and efficient DSP resource sharing with constant latency and initiation interval of one across all datatypes. Evaluated on an AMD Xilinx U55c FPGA, XtraMAC achieves 1.4-2.0x higher compute density, reduces per-operation LUT, FF, and DSP consumption by 27-51%, and delivers up to 1.9x greater energy efficiency and 1.2x speedup on representative mixed-precision LLM workloads. The implementation of XtraMAC is open-sourced at https://github.com/Xtra-Computing/XtraMAC.

Problem

• mixed-precision
• LLM inference
• FPGA
• MAC architecture
• datatype adaptability

Innovation

• mixed-precision
• FPGA
• MAC architecture
• datatype-adaptive
• DSP resource sharing