Distributed Generalized Linear Models: A Privacy-Preserving Approach

2025-03-19

AI Summary

Federated training of generalized linear models (GLMs) faces challenges including data privacy leakage, low statistical efficiency, and poor cross-institutional scalability. Method: We propose the first privacy-preserving distributed GLM training framework supporting streaming data and heterogeneous data distributions. Our approach extends privacy-preserving linear regression to the entire GLM family by integrating secure multi-party computation (SMPC) with iterative reweighted least squares (IRLS), while incorporating either differential privacy (DP) or secure aggregation at the sufficient statistic or gradient level. Contribution/Results: The framework provides rigorous ε-differential privacy guarantees. Empirical evaluation demonstrates that its estimation accuracy matches centralized maximum likelihood estimation (MLE), while reducing communication overhead by 40%. This significantly enhances the practicality, security, and generalizability of federated GLM modeling.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Security and privacy of machine learning and AI applications
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

This paper presents a novel approach to classical linear regression, enabling model computation from data streams or in a distributed setting while preserving data privacy in federated environments. We extend this framework to generalized linear models (GLMs), ensuring scalability and adaptability to diverse data distributions while maintaining privacy-preserving properties. To assess the effectiveness of our approach, we conduct numerical studies on both simulated and real datasets, comparing our method with conventional maximum likelihood estimation for GLMs using iteratively reweighted least squares. Our results demonstrate the advantages of the proposed method in distributed and federated settings.

Problem

• Privacy-preserving distributed linear regression in federated environments.
• Extending framework to generalized linear models for scalability.
• Comparing method with traditional GLM estimation using real datasets.

Innovation

• Privacy-preserving distributed linear regression
• Scalable generalized linear models framework
• Comparison with maximum likelihood estimation