CB-SpMV:A Data Aggregating and Balance Algorithm for Cache-Friendly Block-Based SpMV on GPUs

2026-05-18

AI Summary

This work addresses the poor cache locality, low hardware utilization, and load imbalance commonly observed in sparse matrix-vector multiplication (SpMV) on GPUs by proposing the CB-SpMV algorithm. The method introduces an adaptive two-dimensional blocking scheme that enhances intra-block data locality through virtual pointer aggregation and integrates column aggregation, dynamic sub-block format selection, and cross-block load balancing. Experimental evaluation on 2,843 SuiteSparse matrices demonstrates that CB-SpMV achieves an average speedup of 3.95× over state-of-the-art approaches such as cuSPARSE-BSR on both NVIDIA A100 and RTX 4090 GPUs, significantly improving cache hit rates and computational throughput.

Abstract

Sparse matrix-vector multiplication (SpMV) is crucial in computational science, engineering, and machine learning. Despite substantial efforts to improve SpMV performance on GPUs through various techniques, issues related to data locality, hardware utilization, and load balancing persist, leaving room for further optimization. This paper presents CB-SpMV, a cache-friendly SpMV optimization algorithm, using a novel data convergent and adaptable 2D blocking structure. The matrix in CB-SpMV is divided into independent sub-blocks, with virtual pointers aggregating different types of intra-block data for better cache-level data locality. To enhance hardware utilization, a block-aware column aggregation strategy and the selection of sub-block formats are proposed to accelerate computation and adapt to varying sparse matrices. Finally, an inter-block load-balancing algorithm is designed to ensure efficient workload distribution across thread blocks. Experimental evaluations on 2,843 matrices from the SuiteSparse Collection show that CB-SpMV significantly improves cache hit rates and achieves average speedups of up to 3.95x over state-of-the-art methods like cuSPARSE-BSR, TileSpMV, and DASP on NVIDIA A100 and RTX 4090 GPUs. The implementation is available at: \url{https://github.com/xing-cong/CB-Sparse}.

Problem

• SpMV
• data locality
• hardware utilization
• load balancing
• GPU

Innovation

• cache-friendly
• 2D blocking
• data aggregation
• load balancing
• SpMV