Jack of All Scales: A Versatile FPGA Tensor Block for MXFP Precisions

2026-07-15

AI Summary

This work addresses the lack of native support for MXFP low-precision floating-point formats (e.g., MXFP8/6/4) in existing FPGA DSP blocks, which limits energy efficiency and computational density for deep learning inference. Focusing on the Intel Agilex-5 FPGA, the study proposes an architecture-compatible enhancement to the DSP tensor mode that, for the first time, enables unified native support across the full MXFP format spectrum. Leveraging the open-source ASAP7 PDK, a streamlined DSP core is developed by integrating soft logic with a custom tensor architecture. This design achieves an average 4.2× throughput improvement in matrix multiplication across all MXFP formats while incurring only a 1.8% increase in chip area, thereby bridging a critical gap in FPGA hardware support for MXFP arithmetic.

Abstract

Modern deep learning workloads increasingly rely on narrow numerical formats to improve efficiency and reduce memory footprint. The recently standardized microscaling floating-point (MXFP) family of formats, including MXFP8, MXFP6, and MXFP4, offers a practical approach to low-precision inference, yet the digital signal processing (DSP) blocks in current FPGA architectures offer limited native support for these formats. In this work, we first present a comprehensive characterization of MXFP dot product implementations on Altera Agilex-5 FPGAs, exploring a range of strategies spanning pure soft logic, DSP blocks in fixed-point, floating-point, and tensor modes. Our results show that while the tensor mode delivers the highest arithmetic density for MXFP4 (E2M1) and MXFP6 (E2M3), it cannot implement MXFP6 (E3M2) or any MXFP8 precisions, forcing designers to fall back to lower-density alternatives. Motivated by this gap, we propose targeted modifications to the DSP block's internal tensor-mode architecture that enable native support for all MXFP precisions while retaining backward compatibility. We estimate the area cost of these modifications using a simplified version of the Agilex-5 DSP block core implemented using the open-source ASAP7 PDK. We evaluate a variety of modified DSP block designs that present a tradeoff between format coverage, arithmetic density, and area overhead. Our preferred design point increases the DSP tile area by 36%, corresponding to only 1.8\% of the total FPGA die area. We evaluate the device-level impact of our enhanced DSP block by comparing systolic array matrix multiplier implementations across all MXFP precisions, contrasting the best-available strategies on the existing architecture against designs leveraging our modified DSP block. Our results demonstrate an average throughput improvement of 4.2x across all supported MXFP formats.

Problem

• MXFP
• FPGA
• DSP block
• tensor mode
• low-precision inference

Innovation

• MXFP
• FPGA
• DSP block
• tensor mode
• low-precision inference