Ocean: Fast Estimation-Based Sparse General Matrix-Matrix Multiplication on GPU

2026-04-20

AI Summary

This work addresses the performance bottlenecks of sparse general matrix-matrix multiplication (SpGEMM) on GPUs, which stem from irregular memory access patterns and load imbalance, as well as the substantial overhead of symbolic computation in conventional two-phase approaches. The authors propose an estimation-based SpGEMM execution framework that eliminates the exact symbolic phase entirely. It employs a lightweight HyperLogLog estimator to predict the output sparsity structure and integrates a dynamic workflow selection mechanism with a hybrid hash accumulator that fuses shared and global memory. Evaluated on NVIDIA A100 and H100 GPUs across diverse sparse matrices, the proposed method achieves speedups of 1.4× to 2.8× over state-of-the-art GPU SpGEMM libraries, establishing the first highly efficient SpGEMM framework that completely bypasses symbolic computation.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements

Abstract

In computational science and data analytics, many workloads involve irregular and sparse computations that are inherently difficult to optimize for modern hardware. A key kernel is Sparse General Matrix-Matrix Multiplication (SpGEMM), which underpins simulations, graph analytics, and machine learning applications. SpGEMM exhibits irregular memory access patterns and workload imbalance, making it challenging to achieve high performance on GPUs. Current GPU SpGEMM solutions typically rely on a two-pass workflow to address load imbalance and reduce memory access. The symbolic pass, which determines the number of output elements per row, accounts for roughly 28% of the total runtime on average. In this work, we question the necessity of exact symbolic computation and introduce an estimation-based SpGEMM workflow. Our approach replaces the costly symbolic step with lightweight HyperLogLog estimators, combined with an analysis strategy that dynamically selects the optimal workflow and guides accumulator configuration. In addition, we introduce a hybrid accumulator design, including a novel hash-based accumulator that leverages both shared and global memory. Our approach consistently outperforms leading GPU SpGEMM implementations across a wide range of both square and rectangular matrices, achieving speedups of 1.4x-2.8x on NVIDIA A100 and H100 architectures.

Problem

• SpGEMM
• GPU
• sparse matrix multiplication
• load imbalance
• irregular memory access

Innovation

• SpGEMM
• estimation-based
• HyperLogLog
• hybrid accumulator
• GPU optimization