Multiple Double Arithmetic on NVIDIA Tensor Cores

2026-07-07

AI Summary

This work addresses the challenge of efficiently executing multiple double arithmetic on NVIDIA Ampere A100 Tensor Core GPUs, which, despite supporting FP64 matrix operations, exhibit poor performance in the presence of branching instructions—particularly during the normalization step inherent to multiple double arithmetic. To overcome this limitation, the study presents the first adaptation of Ozaki-style branch-free algorithms to this hardware platform, effectively eliminating conditional branches in normalization and thereby enabling high-performance multiple double computations. Experimental results demonstrate that the proposed approach yields significant performance improvements over conventional implementations. The accompanying implementation has been made publicly available under the GPU GPL license.

Abstract

A multiple double is an unevaluated sum of doubles. An NVIDIA tensor core is a specialized high performance compute core for matrix multiplication. The Ampere A100, released in 2020, introduced tensor cores capable of 64-bit floating-point arithmetic. Every multiple double arithmetical operation requires renormalization, which involves branching, for which tensor cores are unsuited. To solve this problem caused by renormalization, we apply a solution similar to the Ozaki scheme [Ozaki et al, Numerical Algorithms, 2012]. Our software is available under the GPU GPL license on github.

Problem

• multiple double
• Tensor Cores
• renormalization
• branching
• 64-bit floating-point

Innovation

• multiple double arithmetic
• Tensor Cores
• renormalization
• Ozaki scheme
• high-precision computing