Exceeding the Numerical and Performance Characteristics of IEEE-754 SGEMM with BFloat16 Tensor Cores on GPUs for Scientific Computing

2026-05-15

AI Summary

This work proposes an efficient bfloat16 (BF16)-based emulation approach leveraging Tensor Cores to meet the high-performance demands of single-precision (FP32) matrix multiplication in scientific computing. By integrating FP32 accumulators, dedicated scaling hardware native to the Blackwell architecture, and comprehensive support for subnormal numbers, the method achieves substantial gains in both performance and energy efficiency while preserving high numerical accuracy. We present the first implementation on Blackwell GPUs that simultaneously optimizes accuracy, speed, and energy efficiency for BF16-emulated FP32 general matrix-matrix multiplication (GEMM), outperforming native FP32 SGEMM across all metrics. This advancement establishes a superior low-precision acceleration strategy for scientific applications requiring FP32-level fidelity.

Abstract

Largely due to their increased native capacity for numerical intensity and power efficiency, reduced-precision floating-point computing resources, primarily used in artificial intelligence (AI) applications, have expanded at a greater rate than their higher-precision relatives. This has led to various efforts focused upon leveraging plentiful reduced-precision hardware to mimic higher-precision mathematical calculations. This paper studies a specific use case, namely the use of bfloat16 (BF16) Tensor Cores found on modern GPUs in service of single precision (FP32) matrix multiply operations. Given that BF16 and FP32 share the same dynamic range, the option to accumulate BF16 operations into FP32 accumulators (at full-speed), and additional BF16 arithmetic characteristics specific to the Blackwell GPU architecture, such as integrated scaling hardware, such emulation is highly motivated. This paper examines the performance, efficiency, power, and numerical characteristics of FP32 matrix multiplication via BF16-based emulation and demonstrates how it exceeds numerical and performance characteristics of native FP32 for scientific applications. We also discuss a full library-ready implementation that correctly deals with denormals.

Problem

• bfloat16
• Tensor Cores
• FP32 matrix multiplication
• scientific computing
• reduced-precision computing

Innovation

• bfloat16
• Tensor Cores
• FP32 emulation
• scientific computing
• Blackwell GPU