An Adaptive Differentially Private Federated Learning Framework with Bi-level Optimization

2026-02-06

AI Summary

This work addresses the challenge of gradient instability in federated learning under device heterogeneity and non-IID data, which is exacerbated by conventional differential privacy mechanisms that introduce excessive perturbation, leading to training oscillation and performance degradation. To mitigate this, the authors propose an adaptive differentially private federated learning framework featuring a lightweight client-side compression module to constrain gradient variance, a server-side dynamic clipping threshold adjustment strategy, and a constraint-aware aggregation mechanism. Integrating a bilevel optimization structure, adaptive gradient clipping, local representation regularization, and noise-aware aggregation, the method significantly enhances convergence stability and model accuracy in heterogeneous settings while preserving rigorous differential privacy guarantees. Experimental results on CIFAR-10 and SVHN demonstrate that the framework effectively alleviates the performance degradation typically induced by privacy-preserving mechanisms.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Bias, Fairness & Privacy
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity, non-independent, and identically distributed (Non-IID) data often lead to highly unstable and biased gradient updates. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and performance degradation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local compressed module is introduced to regularize intermediate representations and constrain gradient variability, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10 and SVHN demonstrate improved convergence stability and classification accuracy.

Problem

• federated learning
• differential privacy
• device heterogeneity
• Non-IID data
• gradient perturbation

Innovation

• adaptive differential privacy
• federated learning
• gradient clipping
• non-IID data
• constraint-aware aggregation