Private Rate-Constrained Optimization with Applications to Fair Learning

2025-05-28

AI Summary

Under differential privacy (DP), fairness constraints—such as demographic parity and equal opportunity—induce non-decomposable objective functions, rendering standard DP-SGD inapplicable. This work is the first to incorporate group-fairness rate constraints into the DP optimization framework. We propose RaCO-DP, a novel algorithm that privatizes constraint gradients via differentially private minibatch histograms, thereby eliminating reliance on loss decomposability. RaCO-DP integrates stochastic gradient descent-ascent (SGDA) with Lagrangian duality, and we provide rigorous theoretical convergence guarantees. Empirically, RaCO-DP achieves Pareto-optimal trade-offs among privacy, fairness, and utility across diverse benchmarks, consistently outperforming existing differentially private fair learning methods.

Technology Category

• Machine Learning: Privacy
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Data transparency and provenance

Abstract

Many problems in trustworthy ML can be formulated as minimization of the model error under constraints on the prediction rates of the model for suitably-chosen marginals, including most group fairness constraints (demographic parity, equality of odds, etc.). In this work, we study such constrained minimization problems under differential privacy (DP). Standard DP optimization techniques like DP-SGD rely on the loss function's decomposability into per-sample contributions. However, rate constraints introduce inter-sample dependencies, violating the decomposability requirement. To address this, we develop RaCO-DP, a DP variant of the Stochastic Gradient Descent-Ascent (SGDA) algorithm which solves the Lagrangian formulation of rate constraint problems. We demonstrate that the additional privacy cost of incorporating these constraints reduces to privately estimating a histogram over the mini-batch at each optimization step. We prove the convergence of our algorithm through a novel analysis of SGDA that leverages the linear structure of the dual parameter. Finally, empirical results on learning under group fairness constraints demonstrate that our method Pareto-dominates existing private learning approaches in fairness-utility trade-offs.

Problem

• Minimizing model error under rate constraints for fair ML
• Solving constrained optimization under differential privacy (DP)
• Addressing inter-sample dependencies in DP rate-constrained problems

Innovation

• DP variant of SGDA for rate constraints
• Private histogram estimation for constraints
• Convergence via linear dual parameter analysis