Manifold-based Algorithms for the Hadamard Decomposition

2026-05-27

AI Summary

This work addresses the Hadamard decomposition problem—recovering two low-rank matrices whose element-wise product approximates a given matrix. By reformulating the problem as a structured matrix factorization with explicit constraints, the authors propose three efficient algorithms: a direct decomposition method based on Manopt, a block projected gradient approach, and a projection-free Riemannian gradient descent scheme. A novel initialization strategy is also introduced to enhance solution accuracy. The proposed methods are particularly well-suited for large-scale sparse data and demonstrate significant improvements over truncated SVD and existing Hadamard decomposition techniques on both synthetic and real-world datasets, confirming their computational efficiency and competitive performance.

Abstract

Given a matrix $X$, and two ranks $r_1$ and $r_2$, the Hadamard decomposition (HD) looks for two low-rank matrices, $X_1$ of rank $r_1$ and $X_2$ of rank $r_2$, both of the same size as $X$, such that $X\approx X_1\circ X_2$, where $\circ$ is the Hadamard (element-wise) product. In most cases, HD is more expressive than standard low-rank approximations such as the truncated singular value decomposition (TSVD), as it can represent higher-rank matrices with the same number of parameters; this is because the rank of $X_1 \circ X_2$ is generically equal to $r_1 r_2$. In this paper, we first present some theoretical insights for HD, in particular a useful reformulation $X\approx WH^\top$ where $W$ and $H$ have $r_1 r_2$ columns and belong to certain manifolds. These allow us to develop three new algorithms for computing HD. The first one uses the representation $X\approx X_1\circ X_2$ and relies on the Manopt toolbox. The other two rely on the reformulation $X\approx WH^\top$: one is a block projected gradient method, and the other is a manifold-based gradient descent algorithm that does not require projection onto the feasible set. The last two algorithms are particularly effective for handling large sparse data. We also propose new initializations that allow us to improve the accuracy of the HD. We compare our algorithms and initialization strategies with the TSVD and with the state of the art. Numerical results show that the new methods are efficient and competitive on both synthetic and real data.

Problem

• Hadamard decomposition
• low-rank approximation
• matrix factorization
• manifold optimization
• element-wise product

Innovation

• Hadamard decomposition
• manifold optimization
• low-rank approximation
• projected gradient
• matrix factorization