Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM

2026-05-05

AI Summary

Standard differential privacy imposes uniform constraints across all features, disregarding the heterogeneity between sensitive and non-sensitive features as well as their interdependencies. This work proposes CorrDP, a novel framework that, for the first time, incorporates feature correlations into differential privacy modeling by quantifying them via total variation distance and providing an estimable mechanism when such correlations are unknown. Building on this, the authors design a correlation-aware gradient perturbation algorithm integrated into the differentially private empirical risk minimization (DP-ERM) pipeline. Experimental results demonstrate that in settings involving non-sensitive features, CorrDP substantially outperforms standard approaches, achieving a superior privacy-utility trade-off while rigorously preserving the privacy of sensitive features.

Abstract

Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, $\textsf{CorrDP}$, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the $\textsf{CorrDP}$ framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that $\textsf{CorrDP}$-based DP-ERM algorithms consistently outperform the standard DP framework in the presence of insensitive features.

Problem

• differential privacy
• feature correlation
• sensitive features
• privacy constraints
• insensitive features

Innovation

• CorrDP
• feature correlation
• differential privacy
• DP-ERM
• total variation distance