Bridging the Gap between Sparse Matrix Reordering and Factorization: A Deep Learning Framework for Fill-in Reduction

2026-05-17

AI Summary

This work addresses the challenge of sparse matrix reordering, which requires predicting fill-in elements prior to factorization—a task rendered intractable by the NP-hard nature of the minimum fill-in ordering problem, leading to a disconnect between reordering objectives and factorization outcomes. To bridge this gap, the authors propose a novel deep learning framework that integrates spectral embedding with a multi-scale graph neural network. The approach leverages an approximation of the smallest eigenvector of the graph Laplacian to capture global structural information, constructs a surrogate function for fill-in prediction, and optimizes the latent fill-in space using rank-based distribution modeling. This method effectively reconciles the semantic and objective discrepancies between reordering and factorization, achieving fill-reduction performance on par with or superior to both classical graph-theoretic algorithms and existing deep learning approaches across multiple benchmarks.

Abstract

Sparse matrix reordering can significantly reduce the fill-in during matrix factorization, thereby decreasing the computational and storage requirements in sparse matrix computations. Finding a minimal fill-in ordering is known to be an NP-hard problem. Moreover, there is a paradox: matrix reordering is applied before matrix factorization, but fill-ins that matrix reordering methods aim at are generated from matrix factorization. To bridge the gap between reordering and factorization, we propose a deep learning framework to minimize a fill-in surrogate function based on spectral embedding. First, we employ a multi-grid-like GNN architecture to learn to approximate the smallest eigenvectors of its graph Laplacian matrix, i.e. spectral embedding, and capture the global structural information of the matrix. Then, another multi-grid-like GNN architecture is used to minimize the potential space where fill-in can occur based on the rank distribution. Experimental results indicate that our approach achieves competitive performance compared with traditional graph-theoretic algorithms and deep learning methods.

Problem

• sparse matrix reordering
• fill-in reduction
• matrix factorization
• NP-hard problem
• spectral embedding

Innovation

• deep learning
• sparse matrix reordering
• fill-in reduction
• graph neural network
• spectral embedding