Differential Privacy in Two-Layer Networks: How DP-SGD Harms Fairness and Robustness

2026-03-05

AI Summary

This work addresses the performance degradation, fairness deterioration, and reduced adversarial robustness commonly observed in differentially private training, whose underlying mechanisms in non-convex neural networks remain poorly understood. We propose a unified feature-centering framework to theoretically analyze the feature learning dynamics of DP-SGD in two-layer ReLU convolutional networks. For the first time from a feature learning perspective, we reveal that privacy-preserving noise induces fairness and robustness issues through an imbalance in the feature-to-noise ratio (FNR), particularly affecting inter-class separation and long-tailed samples. We derive a bound on test loss and validate on both synthetic and real-world data that FNR imbalance leads to suboptimal feature learning, thereby exacerbating group disparities and adversarial vulnerability. Our findings also cast doubt on the universal efficacy of the “public pretraining + private finetuning” paradigm.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Differentially private learning is essential for training models on sensitive data, but empirical studies consistently show that it can degrade performance, introduce fairness issues like disparate impact, and reduce adversarial robustness. The theoretical underpinnings of these phenomena in modern, non-convex neural networks remain largely unexplored. This paper introduces a unified feature-centric framework to analyze the feature learning dynamics of differentially private stochastic gradient descent (DP-SGD) in two-layer ReLU convolutional neural networks. Our analysis establishes test loss bounds governed by a crucial metric: the feature-to-noise ratio (FNR). We demonstrate that the noise required for privacy leads to suboptimal feature learning, and specifically show that: 1) imbalanced FNRs across classes and subpopulations cause disparate impact; 2) even in the same class, noise has a greater negative impact on semantically long-tailed data; and 3) noise injection exacerbates vulnerability to adversarial attacks. Furthermore, our analysis reveals that the popular paradigm of public pre-training and private fine-tuning does not guarantee improvement, particularly under significant feature distribution shifts between datasets. Experiments on synthetic and real-world data corroborate our theoretical findings.

Problem

• Differential Privacy
• Fairness
• Robustness
• Feature Learning
• DP-SGD

Innovation

• Differential Privacy
• Feature-to-Noise Ratio
• DP-SGD
• Fairness
• Adversarial Robustness