Fast Decentralized Federated Low Rank Matrix Recovery from Column-wise Linear Projections

2022-04-18

AI Summary

This paper addresses the problem of efficiently reconstructing a low-rank matrix from column-wise linear projections acquired independently by nodes in a fully decentralized edge system—where no central coordinator exists, communication is bandwidth-constrained, and the low-rank constraint induces nonconvexity. To this end, we propose the first provably linearly convergent decentralized alternating projection gradient descent (D-APGD) algorithm. D-APGD jointly enforces the low-rank constraint and distributed optimization via local singular value thresholding and iterative gradient exchange over the communication graph. Under standard graph connectivity assumptions, we establish theoretical guarantees of linear convergence. Experiments demonstrate that D-APGD reduces communication overhead by over 40% compared to state-of-the-art distributed matrix completion and compressed sensing methods, while achieving superior reconstruction accuracy and faster convergence. The framework provides a new paradigm for resource-efficient intelligent inference at the network edge.

Abstract

This work develops a provably accurate fully-decentralized alternating projected gradient descent (GD) algorithm for recovering a low rank (LR) matrix from mutually independent projections of each of its columns, in a fast and communication-efficient fashion. To our best knowledge, this work is the first attempt to develop a provably correct decentralized algorithm (i) for any problem involving the use of an alternating projected GD algorithm; (ii) and for any problem in which the constraint set to be projected to is a non-convex set.

Problem

• Decentralized algorithm for low-rank matrix recovery
• Communication-efficient multi-task representation learning
• Non-convex alternating projected gradient descent method

Innovation

• Decentralized alternating projected gradient descent algorithm
• Recovering low rank matrix from projections
• Non-convex constraint set projection method