Accelerating Differentially Private Federated Learning via Adaptive Extrapolation

2025-04-14

AI Summary

Differential privacy federated learning (DP-FL) suffers from slow convergence under data heterogeneity, while existing acceleration methods either introduce additional hyperparameters or increase client-side computational overhead. Method: We propose DP-FedEXP—the first adaptive extrapolation algorithm for DP-FL that requires no new hyperparameters and incurs zero additional computation at clients. Its core innovation is a dynamic global step-size adjustment mechanism guided by the diversity of local model updates, enabling accelerated convergence under differential privacy constraints. Contribution/Results: We theoretically prove that DP-FedEXP improves convergence rate while satisfying $(varepsilon,delta)$-differential privacy. Empirically, on multiple heterogeneous benchmarks, it reduces communication rounds by up to 35% compared to state-of-the-art DP-FL methods, achieves higher test accuracy, and eliminates privacy leakage risks associated with hyperparameter tuning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The federated learning (FL) framework enables multiple clients to collaboratively train machine learning models without sharing their raw data, but it remains vulnerable to privacy attacks. One promising approach is to incorporate differential privacy (DP)-a formal notion of privacy-into the FL framework. DP-FedAvg is one of the most popular algorithms for DP-FL, but it is known to suffer from the slow convergence in the presence of heterogeneity among clients' data. Most of the existing methods to accelerate DP-FL require 1) additional hyperparameters or 2) additional computational cost for clients, which is not desirable since 1) hyperparameter tuning is computationally expensive and data-dependent choice of hyperparameters raises the risk of privacy leakage, and 2) clients are often resource-constrained. To address this issue, we propose DP-FedEXP, which adaptively selects the global step size based on the diversity of the local updates without requiring any additional hyperparameters or client computational cost. We show that DP-FedEXP provably accelerates the convergence of DP-FedAvg and it empirically outperforms existing methods tailored for DP-FL.

Problem

• Improving convergence speed in DP-FedAvg with client data heterogeneity
• Eliminating extra hyperparameters to avoid privacy leakage risks
• Reducing client computational costs in DP-FL frameworks

Innovation

• Adaptive global step size selection
• No additional hyperparameters needed
• No extra client computational cost