Federated Learning with Differential Privacy: An Utility-Enhanced Approach

2025-03-27

AI Summary

Federated learning preserves data locality but remains vulnerable to privacy leakage via model updates; while differential privacy (DP) offers rigorous user- or sample-level guarantees, it often incurs substantial utility degradation. This paper proposes an efficient DP-Fed framework integrating Haar wavelet transform with a novel noise-injection mechanism. Specifically, orthogonal wavelet decomposition is applied during client-side gradient compression to reduce the effective dimensionality sensitive to noise, and a variance-asymptotically optimal noise injection strategy is designed to jointly ensure strict privacy and fast convergence. We theoretically establish ε-differential privacy and an O(1/T) convergence rate. Experiments on multiple real-world datasets demonstrate that, under identical privacy budgets, our method improves model accuracy by up to 12.6% over state-of-the-art DP-Fed approaches, accelerates convergence, and significantly enhances the privacy–utility trade-off.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Federated learning has emerged as an attractive approach to protect data privacy by eliminating the need for sharing clients' data while reducing communication costs compared with centralized machine learning algorithms. However, recent studies have shown that federated learning alone does not guarantee privacy, as private data may still be inferred from the uploaded parameters to the central server. In order to successfully avoid data leakage, adopting differential privacy (DP) in the local optimization process or in the local update aggregation process has emerged as two feasible ways for achieving sample-level or user-level privacy guarantees respectively, in federated learning models. However, compared to their non-private equivalents, these approaches suffer from a poor utility. To improve the privacy-utility trade-off, we present a modification to these vanilla differentially private algorithms based on a Haar wavelet transformation step and a novel noise injection scheme that significantly lowers the asymptotic bound of the noise variance. We also present a holistic convergence analysis of our proposed algorithm, showing that our method yields better convergence performance than the vanilla DP algorithms. Numerical experiments on real-world datasets demonstrate that our method outperforms existing approaches in model utility while maintaining the same privacy guarantees.

Problem

• Enhancing privacy-utility trade-off in federated learning
• Reducing noise variance in differential privacy mechanisms
• Improving convergence performance of private federated algorithms

Innovation

• Haar wavelet transformation enhances privacy
• Novel noise injection reduces noise variance
• Better convergence than vanilla DP algorithms