Robust and Differentially Private PCA for non-Gaussian data

2025-07-21

AI Summary

Existing differentially private PCA methods exhibit weak robustness against non-Gaussian and contaminated data, often relying on strong distributional assumptions (e.g., sub-Gaussianity) or requiring accurate estimation of unknown parameters. Method: This paper proposes a robust differentially private PCA algorithm based on bounded transformations. Leveraging the invariance of covariance eigenstructure under elliptical distributions, it integrates bounded transformations with differential privacy mechanisms without assuming sub-Gaussianity or requiring parameter estimation. Contribution/Results: Theoretically, the method achieves consistent subspace recovery. Empirically, it significantly outperforms state-of-the-art approaches in statistical utility and stability under heavy-tailed distributions, adversarial noise, and data contamination, while improving computational efficiency.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Recent advances have sparked significant interest in the development of privacy-preserving Principal Component Analysis (PCA). However, many existing approaches rely on restrictive assumptions, such as assuming sub-Gaussian data or being vulnerable to data contamination. Additionally, some methods are computationally expensive or depend on unknown model parameters that must be estimated, limiting their accessibility for data analysts seeking privacy-preserving PCA. In this paper, we propose a differentially private PCA method applicable to heavy-tailed and potentially contaminated data. Our approach leverages the property that the covariance matrix of properly rescaled data preserves eigenvectors and their order under elliptical distributions, which include Gaussian and heavy-tailed distributions. By applying a bounded transformation, we enable straightforward computation of principal components in a differentially private manner. Additionally, boundedness guarantees robustness against data contamination. We conduct both theoretical analysis and empirical evaluations of the proposed method, focusing on its ability to recover the subspace spanned by the leading principal components. Extensive numerical experiments demonstrate that our method consistently outperforms existing approaches in terms of statistical utility, particularly in non-Gaussian or contaminated data settings.

Problem

• Develops private PCA for heavy-tailed and contaminated data
• Ensures robustness and privacy without restrictive assumptions
• Improves subspace recovery in non-Gaussian settings

Innovation

• Differentially private PCA for heavy-tailed data
• Bounded transformation ensures robustness and privacy
• Preserves eigenvectors under elliptical distributions