Self-Supervised Learning for Sparse Matrix Reordering

2026-05-17

AI Summary

This work addresses the NP-complete problem of minimizing fill-in during sparse matrix reordering to reduce memory and computational costs in matrix factorization. It introduces, for the first time, an end-to-end framework that reformulates the Fill-Path theorem into a learnable self-supervised objective based on path-based triplet inequalities. The approach employs a multi-scale graph neural network to extract structural vertex features and incorporates a tailored triplet sampling strategy along with a max-chain loss function, thereby avoiding reliance on proxy objectives lacking theoretical guarantees. Evaluated on the SuiteSparse dataset, the method significantly reduces fill-in and accelerates LU decomposition, outperforming both classical graph-theoretic and contemporary deep learning approaches.

Abstract

Rearranging the rows or columns of a sparse matrix using an appropriate ordering can significantly reduce fill-ins, i.e., new nonzeros introduced during matrix factorization, decreasing memory usage and runtime. However, finding an ordering that minimizes fill-ins is NP-complete. Existing approaches, including graph-theoretic and deep learning methods, rely on surrogate objectives without theoretical guarantees. The Fill-Path Theorem reveals a direct and intrinsic relationship between fill-in generation and the sparse structure of the matrix as path triplet inequalities. Here we first employ a multigrid graph network to capture structural information for each vertex. We then derive a triplet sampling strategy based on inequalities. Finally, we introduce an end-max chain loss function to reduce the number of triplets whose predicted scores satisfy these inequalities. Experimental evaluations on the publicly available SuiteSparse matrix collection demonstrate the superiority of the proposed method in terms of both fill-in reduction and speedup in LU factorization time.

Problem

• Sparse Matrix Reordering
• Fill-in Reduction
• Self-Supervised Learning
• Matrix Factorization
• NP-complete

Innovation

• Self-Supervised Learning
• Sparse Matrix Reordering
• Fill-In Reduction
• Multigrid Graph Network
• Triplet Inequality Sampling