Tight Differentially Private PCA via Matrix Coherence

2025-10-30

AI Summary

This paper addresses the problem of computing the subspace spanned by the top-$r$ singular vectors of a matrix under differential privacy. We propose an efficient algorithm based on truncated SVD followed by Gaussian perturbation. Its estimation error depends solely on the rank-$r$ coherence and the spectral gap—introducing rank-$r$ coherence into private PCA analysis for the first time—and we prove that this coherence measure is invariant under Gaussian perturbation, thereby resolving an open problem posed by Hardt and Roth. Theoretically, our algorithm achieves the same statistical accuracy as the optimal non-private method for dense matrices. In the single-spike Wishart model, it matches the performance of the optimal non-private algorithm and significantly outperforms existing differentially private approaches. Furthermore, our framework extends naturally to graph-constrained optimization problems, including Max-Cut.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We revisit the task of computing the span of the top $r$ singular vectors $u_1, ldots, u_r$ of a matrix under differential privacy. We show that a simple and efficient algorithm -- based on singular value decomposition and standard perturbation mechanisms -- returns a private rank-$r$ approximation whose error depends only on the emph{rank-$r$ coherence} of $u_1, ldots, u_r$ and the spectral gap $σ_r - σ_{r+1}$. This resolves a question posed by Hardt and Roth~cite{hardt2013beyond}. Our estimator outperforms the state of the art -- significantly so in some regimes. In particular, we show that in the dense setting, it achieves the same guarantees for single-spike PCA in the Wishart model as those attained by optimal non-private algorithms, whereas prior private algorithms failed to do so. In addition, we prove that (rank-$r$) coherence does not increase under Gaussian perturbations. This implies that any estimator based on the Gaussian mechanism -- including ours -- preserves the coherence of the input. We conjecture that similar behavior holds for other structured models, including planted problems in graphs. We also explore applications of coherence to graph problems. In particular, we present a differentially private algorithm for Max-Cut and other constraint satisfaction problems under low coherence assumptions.

Problem

• Computing top singular vectors span under differential privacy
• Improving private PCA accuracy via matrix coherence analysis
• Developing private algorithms for graph problems using coherence

Innovation

• Private PCA via matrix coherence analysis
• SVD with Gaussian perturbation mechanisms
• Coherence preservation under Gaussian noise injection