Riemannian Federated Learning via Averaging Gradient Stream

2024-09-11

arXiv.org

AI Summary

Motivated by the lack of research on federated learning over Riemannian manifolds—and particularly the absence of frameworks addressing partial client participation and data heterogeneity—this paper proposes Riemannian Federated Averaged Gradient Flow (RFedAGS), the first systematic extension of the FedAvg paradigm to non-Euclidean spaces. Methodologically, RFedAGS integrates manifold-valued gradient computation with distributed gradient flow averaging, enabling privacy-preserving collaborative optimization. Theoretically, we establish the first convergence analysis of RFedAGS under both constant and diminishing step sizes: proving sublinear convergence, global convergence, and—under the Riemannian Polyak–Łojasiewicz condition—linear convergence rate. Extensive experiments on synthetic and real-world manifold datasets demonstrate that RFedAGS consistently outperforms existing baselines, validating its efficacy and robustness in heterogeneous, partially participating settings.

Technology Category

• Machine Learning: Learning with Manifolds
• Multiagent Systems: Mechanism Design
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In recent years, federated learning has garnered significant attention as an efficient and privacy-preserving distributed learning paradigm. In the Euclidean setting, Federated Averaging (FedAvg) and its variants are a class of efficient algorithms for expected (empirical) risk minimization. This paper develops and analyzes a Riemannian Federated Averaging Gradient Stream (RFedAGS) algorithm, which is a generalization of FedAvg, to problems defined on a Riemannian manifold. Under standard assumptions, the convergence rate of RFedAGS with fixed step sizes is proven to be sublinear for an approximate stationary solution. If decaying step sizes are used, the global convergence is established. Furthermore, assuming that the objective obeys the Riemannian Polyak-{L}ojasiewicz property, the optimal gaps generated by RFedAGS with fixed step size are linearly decreasing up to a tiny upper bound, meanwhile, if decaying step sizes are used, then the gaps sublinearly vanish. Numerical simulations conducted on synthetic and real-world data demonstrate the performance of the proposed RFedAGS.

Problem

• Extending federated learning to Riemannian manifold settings
• Addressing partial participation in distributed optimization algorithms
• Handling data heterogeneity across agents in federated learning

Innovation

• Riemannian federated learning using gradient streams
• Handles partial participation and data heterogeneity
• Ensures global convergence with sublinear rates