Stabilization of Perturbed Loss Function: Differential Privacy without Gradient Noise

2025-08-21

AI Summary

To address the high computational overhead, poor training stability, and susceptibility to environmental noise caused by gradient perturbation in multi-user Local Differential Privacy (LDP) training, this paper proposes SPOF—a novel LDP training mechanism that eliminates gradient perturbation entirely. Its core innovation lies in the first application of Taylor series expansion to approximate the loss function, followed by LDP-compliant perturbation of the resulting polynomial coefficients; this is combined with noise calibration and a distributed architecture to enable parallel privacy preservation across users in Wireless Body Area Networks (WBANs). By circumventing the gradient-noise injection inherent in DP-SGD, SPOF significantly improves training efficiency and robustness. Experimental results demonstrate that SPOF achieves an average 3.5% improvement in reconstruction accuracy and reduces training time by up to 57.2% compared to DP-SGD, yielding substantial gains in the privacy–utility trade-off.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Local Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Privacy-enhancing technologies
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

We propose SPOF (Stabilization of Perturbed Loss Function), a differentially private training mechanism intended for multi-user local differential privacy (LDP). SPOF perturbs a stabilized Taylor expanded polynomial approximation of a model's training loss function, where each user's data is privatized by calibrated noise added to the coefficients of the polynomial. Unlike gradient-based mechanisms such as differentially private stochastic gradient descent (DP-SGD), SPOF does not require injecting noise into the gradients of the loss function, which improves both computational efficiency and stability. This formulation naturally supports simultaneous privacy guarantees across all users. Moreover, SPOF exhibits robustness to environmental noise during training, maintaining stable performance even when user inputs are corrupted. We compare SPOF with a multi-user extension of DP-SGD, evaluating both methods in a wireless body area network (WBAN) scenario involving heterogeneous user data and stochastic channel noise from body sensors. Our results show that SPOF achieves, on average, up to 3.5% higher reconstruction accuracy and reduces mean training time by up to 57.2% compared to DP-SGD, demonstrating superior privacy-utility trade-offs in multi-user environments.

Problem

• Achieving differential privacy without gradient noise injection
• Providing multi-user local privacy with stabilized loss approximation
• Maintaining robustness against environmental noise during training

Innovation

• Stabilizes Taylor expanded polynomial loss approximation
• Perturbs polynomial coefficients instead of gradients
• Adds calibrated noise to coefficients for privacy