🤖 AI Summary
This work addresses the instability of objective perturbation in high-dimensional LASSO under differential privacy, which arises from heterogeneity in covariate scales and compromises both estimation accuracy and privacy guarantees. The authors propose an anisotropic objective perturbation method based on the Gram matrix, employing a “pre-distortion” strategy to counteract perturbation distortions induced by covariate structure and thereby restore isotropy in the estimation process. Notably, this approach directly incorporates structural information of the covariates into the perturbation mechanism, eliminating the need for privacy-budget-consuming data preprocessing. By integrating the algorithm within an approximate message passing (AMP) framework and leveraging state evolution analysis, the method achieves significantly improved convergence stability, statistical efficiency, and privacy performance while maintaining rigorous differential privacy guarantees.
📝 Abstract
We study high-dimensional LASSO under differential privacy via objective perturbation with heterogeneous covariate scales. In practical scenarios, covariates often exhibit diverse scales; however, standard preprocessing is problematic under privacy constraints, as it consumes additional privacy budget. This heterogeneity induces effective anisotropy in the objective perturbation via the inverse Gram matrix of covariates, which can degrade the stability and accuracy of algorithms. To address this, we propose a Gram-based anisotropic objective perturbation, a ``pre-distortion" strategy that counteracts the distortion from the covariate structure to restore isotropy in the estimation process. Using an Approximate Message Passing (AMP) framework and state evolution analysis, we demonstrate that our proposed perturbation significantly stabilizes convergence and improves both statistical efficiency and privacy performance compared to standard uniform noise injection. Our results provide theoretical insights into designing stable and efficient private estimators without relying on data-dependent preprocessing.