Personalized federated learning, Row-wise fusion regularization, Multivariate modeling, Sparse estimation

2025-10-16

AI Summary

In multivariate response modeling, existing federated learning methods struggle to simultaneously address client-level model heterogeneity and variable-level structural consistency. Method: We propose Sparse Row-wise Fusion (SROF), a novel regularization paradigm that jointly learns row-wise clustering and intra-row sparsity—enabling simultaneous personalization and cross-client structural sharing. Built upon a linearized ADMM framework, we design RowFed: a privacy-preserving, partially participating, communication-efficient algorithm. Contribution/Results: We establish theoretical guarantees showing that SROF possesses the oracle property and that RowFed converges. Empirical evaluations demonstrate that RowFed significantly outperforms baselines—including NonFed and FedAvg—in parameter estimation accuracy, predictive performance, and variable clustering quality.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We study personalized federated learning for multivariate responses where client models are heterogeneous yet share variable-level structure. Existing entry-wise penalties ignore cross-response dependence, while matrix-wise fusion over-couples clients. We propose a Sparse Row-wise Fusion (SROF) regularizer that clusters row vectors across clients and induces within-row sparsity, and we develop RowFed, a communication-efficient federated algorithm that embeds SROF into a linearized ADMM framework with privacy-preserving partial participation. Theoretically, we establish an oracle property for SROF-achieving correct variable-level group recovery with asymptotic normality-and prove convergence of RowFed to a stationary solution. Under random client participation, the iterate gap contracts at a rate that improves with participation probability. Empirically, simulations in heterogeneous regimes show that RowFed consistently lowers estimation and prediction error and strengthens variable-level cluster recovery over NonFed, FedAvg, and a personalized matrix-fusion baseline. A real-data study further corroborates these gains while preserving interpretability. Together, our results position row-wise fusion as an effective and transparent paradigm for large-scale personalized federated multivariate learning, bridging the gap between entry-wise and matrix-wise formulations.

Problem

• Personalized federated learning for heterogeneous client models
• Addressing cross-response dependence with row-wise fusion
• Improving estimation accuracy and cluster recovery efficiency

Innovation

• Row-wise fusion regularizer clusters client row vectors
• Communication-efficient algorithm embeds SROF in ADMM framework
• Method enables privacy-preserving partial client participation