Problem
Research questions and friction points this paper is trying to address.
low-rank methods
deep neural network training
adaptive randomized range finder
implicit Householder QR decomposition
heterogeneous systems
A GPU-Accelerated Blocked Adaptive Randomized Range Finder Based on an Implicit Householder QR Decomposition
🤖 AI Summary
本文提出一种基于隐式Householder QR分解的GPU加速分块自适应随机范围查找方法，用于解决深度神经网络训练中的低秩近似问题。
📝 Abstract
Low-rank methods can reduce the memory and computational requirements of deep neural network training in approaches such as GaLore. Randomized range finders offer an attractive alternative to singular value decompositions, particularly when the required rank is determined adaptively from a prescribed approximation tolerance. We introduce a blocked adaptive randomized range finder based on an implicit Householder QR decomposition and an optimized hybrid CPU--GPU implementation. The proposed method avoids explicit reorthogonalization. Numerical experiments show that it preserves orthogonality and approximation accuracy in regimes where block Gram--Schmidt without reorthogonalization becomes unstable. The blocked formulation exposes matrix--matrix operations and enables overlap of CPU panel factorization with GPU updates. On an NVIDIA GH200, the overlapped implementation reduces the runtime for the largest tested matrix from 9.91 seconds on the CPU to 0.407 seconds. The method provides a stable and efficient building block for low-rank approximation on heterogeneous systems with applications in computational science and engineering.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Blocked Adaptive Randomized Range Finder
Implicit Householder QR Decomposition
Hybrid CPU--GPU Implementation
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
