Online differentially private inference in stochastic gradient descent

2025-05-13

AI Summary

This paper addresses real-time differentially private statistical inference for streaming data in the absence of a trusted data curator. Methodologically, it proposes the first one-pass online stochastic gradient descent (SGD) algorithm satisfying local differential privacy (LDP), enabling real-time parameter estimation and inference without revisiting historical data. The approach introduces an LDP-compliant noise injection mechanism and an adaptive step-size strategy, yielding two tight private confidence intervals. Theoretically, it establishes the optimal convergence rate of the estimator and proves its pathwise weak convergence to a scaled Brownian motion—thereby establishing a functional central limit theorem for LDP-based online inference. Extensive simulations and empirical evaluations on ride-hailing and U.S. insurance datasets demonstrate the algorithm’s statistical validity and practical privacy utility under finite-sample settings.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Local Search

Application Category

• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We propose a general privacy-preserving optimization-based framework for real-time environments without requiring trusted data curators. In particular, we introduce a noisy stochastic gradient descent algorithm for online statistical inference with streaming data under local differential privacy constraints. Unlike existing methods that either disregard privacy protection or require full access to the entire dataset, our proposed algorithm provides rigorous local privacy guarantees for individual-level data. It operates as a one-pass algorithm without re-accessing the historical data, thereby significantly reducing both time and space complexity. We also introduce online private statistical inference by conducting two construction procedures of valid private confidence intervals. We formally establish the convergence rates for the proposed estimators and present a functional central limit theorem to show the averaged solution path of these estimators weakly converges to a rescaled Brownian motion, providing a theoretical foundation for our online inference tool. Numerical simulation experiments demonstrate the finite-sample performance of our proposed procedure, underscoring its efficacy and reliability. Furthermore, we illustrate our method with an analysis of two datasets: the ride-sharing data and the US insurance data, showcasing its practical utility.

Problem

• Online private inference in stochastic gradient descent
• Local differential privacy for streaming data
• One-pass algorithm reducing time and space complexity

Innovation

• Noisy SGD for online private inference
• One-pass algorithm reduces complexity
• Private confidence intervals construction