Private Sketches for Linear Regression

2025-11-10

AI Summary

To address privacy preservation in linear regression over sensitive data, this paper proposes a differentially private (DP) sketching framework—departing from conventional approaches that release noisy parameter estimates. Specifically, we design private sketch generation mechanisms for both least-squares and least-absolute-deviations regression, leveraging matrix random projection combined with calibrated noise injection to preserve key statistical structures of the original data while guaranteeing ε-differential privacy. The resulting sketch is directly compatible with standard non-private solvers without algorithmic modification, significantly enhancing generality and deployment practicality. Theoretically, our estimator achieves optimal convergence rate in estimation error. Empirically, it maintains high accuracy even under strong privacy guarantees (ε ≤ 1), outperforming existing DP regression methods. To the best of our knowledge, this is the first work to systematically construct a reusable, solver-agnostic DP sketch for linear regression.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Linear regression is frequently applied in a variety of domains. In order to improve the efficiency of these methods, various methods have been developed that compute summaries or emph{sketches} of the datasets. Certain domains, however, contain sensitive data which necessitates that the application of these statistical methods does not reveal private information. Differentially private (DP) linear regression methods have been developed for mitigating this problem. These techniques typically involve estimating a noisy version of the parameter vector. Instead, we propose releasing private sketches of the datasets. We present differentially private sketches for the problems of least squares regression, as well as least absolute deviations regression. The availability of these private sketches facilitates the application of commonly available solvers for regression, without the risk of privacy leakage.

Problem

• Developing private sketches for linear regression
• Ensuring differential privacy in dataset summaries
• Applying sketches to least squares and absolute deviations

Innovation

• Private sketches for least squares regression
• Private sketches for least absolute deviations
• Enables use of standard solvers privately