An Exterior Method for Nonnegative Matrix Factorization

2026-05-19

AI Summary

This work addresses the limitations of traditional interior-point methods in non-negative matrix factorization (NMF), which often suffer from slow convergence or suboptimal solutions due to enforcing non-negativity constraints throughout optimization. The authors propose eNMF, an exterior-point framework that decouples low-rank approximation from non-negativity constraints for the first time: it first computes an unconstrained optimal factorization and then maps the factors near the non-negative orthant boundary via factor rotation, followed by an iterative update strategy satisfying the Karush–Kuhn–Tucker (KKT) conditions. This approach offers a geometric perspective on NMF, revealing equivalence classes of factors under permutation and orthogonal transformations and explaining why diverse algorithms often converge to equivalent solutions. Experiments show that eNMF outperforms 81 baseline variants across five datasets, achieving up to 30% lower reconstruction error at equal runtime and up to 150% faster convergence at comparable accuracy, while significantly improving downstream performance in audio and recommendation tasks.

Abstract

Nonnegative matrix factorization (NMF) seeks a low-rank approximation $X \approx UV^T$ with nonnegative factors and is commonly solved using interior methods that enforce feasibility throughout optimization. We show that such constraint-driven approaches can impede progress in the nonconvex landscape, leading to slow convergence or convergence to suboptimal stationary points. We propose an exterior framework for NMF (eNMF) that separates low-rank approximation from nonnegativity enforcement. Our method initializes from the optimal unconstrained factorization and introduces a rotation procedure that maps unconstrained factors to an exterior point closest to the nonnegative orthant. This viewpoint yields an algorithmic framework in which simple iterative updates converge to KKT-satisfying stationary points on the boundary of the positive orthant. The exterior formulation also enables a geometric interpretation of NMF solutions, clarifying equivalence classes of factorizations under permutation and orthogonal transformations. An intriguing numerical result, involving 400 NMF experiments across both real and synthetic datasets, show that in 99% of the cases, different algorithms tend to converge towards equivalent factor matrices. We benchmark eNMF against 9 state-of-the-art NMF algorithms with 9 initialization schemes across 3 real-world and 2 synthetic datasets. eNMF consistently outperforms all 81 competitors, achieving up to 30% lower reconstruction error under equal-time settings and up to 150% speedup under equal-error settings. The downstream experiments further demonstrate substantial performance gains in audio processing and recommendation tasks, corroborating the practical benefits of the proposed exterior optimization framework. Code is available at https://github.com/roychowdhuryresearch/eNMF

Problem

• Nonnegative Matrix Factorization
• Nonconvex Optimization
• Constrained Optimization
• Slow Convergence
• Suboptimal Solutions

Innovation

• exterior optimization
• nonnegative matrix factorization
• rotation-based projection
• KKT convergence
• low-rank approximation