Popcorn: Accelerating Kernel K-means on GPUs through Sparse Linear Algebra

2025-01-09

AI Summary

Kernel K-means suffers from O(n²) computational and memory complexity in kernel matrix construction, rendering it impractical for large-scale datasets. To address this, we reformulate Kernel K-means as a sparse linear algebra problem—its first such formulation—and introduce a GPU-accelerated paradigm based on sparse matrix–matrix (SpMM) and sparse matrix–vector (SpMV) operations. By leveraging the kernel trick for implicit feature mapping and representing key computations via sparse matrices, our approach circumvents dense kernel matrix materialization, thereby reducing both programming complexity and memory footprint. We release the first open-source GPU implementation of Kernel K-means, achieving second-scale clustering on medium-sized datasets: up to 123.8× speedup over CPU-based implementations and 2.6× over non-sparse GPU variants. Our core contribution lies in transforming high-complexity nonlinear clustering into an efficiently parallelizable sparse computation task, establishing a practical pathway for scalable kernel-based clustering.

Technology Category

• Machine Learning: Kernel Methods
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

K-means is a popular clustering algorithm with significant applications in numerous scientific and engineering areas. One drawback of K-means is its inability to identify non-linearly separable clusters, which may lead to inaccurate solutions in certain cases. Kernel K-means is a variant of classical K-means that can find non-linearly separable clusters. However, it scales quadratically with respect to the size of the dataset, taking several minutes to cluster even medium-sized datasets on traditional CPU-based machines. In this paper, we present a formulation of Kernel K-means using sparse-dense matrix multiplication (SpMM) and sparse matrix-vector multiplication (SpMV), and we show that our formulation enables the rapid implementation of a fast GPU-based version of Kernel K-means with little programming effort. Our implementation, named Popcorn, is the first open-source GPU-based implementation of Kernel K-means. Popcorn achieves a speedup of up to 123.8x over a CPU implementation of Kernel K-means and a speedup of up to 2.6x over a GPU implementation of Kernel K-means that does not use sparse matrix computations. Our results support the effectiveness of sparse matrices as tools for efficient parallel programming.

Problem

• Kernel K-means
• large-scale data
• computational efficiency

Innovation

• Kernel K-means acceleration
• Sparse linear algebra operations
• GPU-based computation