Optimizing Semiconductor Device Simulations through Low-Precision Arithmetic

2026-06-24

AI Summary

This work addresses the challenge that traditional FP64 quantum transport simulations cannot leverage the computational advantages of modern GPUs’ low-precision arithmetic, as naive precision reduction often leads to numerical instability. The study systematically analyzes the numerical stability of the Quatrex solver and proposes a precision-aware conversion strategy that balances accuracy and performance. It demonstrates, for the first time, the effective use of low-precision arithmetic in quantum transport simulations that surpass the FP64 exaflop-per-second barrier. By integrating tailored low-precision floating-point formats, stability-preserving mechanisms, and high-performance computing optimizations, the approach achieves a 51% increase in throughput and reduces computational resource consumption by 40% on realistic large-scale benchmarks, all while maintaining solution accuracy.

Abstract

Architectural changes in GPUs, especially the promotion of low-precision computational units, pose significant challenges to traditional, FP64-based high-performance computing (HPC) applications, while also presenting opportunities. Adopting reduced-precision data formats is a promising avenue to exploit the increased throughput capabilities. However, straightforward data conversions may lead to degraded accuracy or even erroneous results. For a given application, only an in-depth analysis of its numerical stability can reveal the potential of low-precision arithmetic. In this work, we consider the open-source quatrex package, a quantum transport solver capable of breaking the sustained FP64 Eflop/s barrier, to illustrate trade-offs between accuracy losses and computational speed-ups when moving from high- to low-precision formats. We use three representative benchmark structures to explore the application's numerical properties. Applying the gained insights to a larger, more realistic system, we achieve up to 51% higher throughput while maintaining accurate results, on 40% fewer HPC resources than the FP64 reference.

Problem

• low-precision arithmetic
• semiconductor device simulation
• numerical stability
• high-performance computing
• accuracy loss

Innovation

• low-precision arithmetic
• semiconductor device simulation
• numerical stability
• quantum transport solver
• throughput optimization