Privacy-Preserving Logistic Regression Training on Large Datasets

2024-06-19

arXiv.org

AI Summary

To address the challenge of privacy-preserving logistic regression training on large-scale encrypted financial data (422K × 200), this paper proposes the first homomorphic encryption (HE) optimization framework integrating curvature-aware second-order gradients. Methodologically: (1) it introduces second-order gradients into HE-based logistic regression for the first time, significantly accelerating convergence; (2) it designs a batched ciphertext training architecture supporting both mini-batch and full-batch scheduling; and (3) it incorporates Nesterov momentum acceleration. Evaluated on real-world encrypted financial datasets, the framework substantially breaks through inherent HE bottlenecks: communication and computation overhead decrease by over 40%, while model accuracy degradation remains below 0.5%. This work establishes a scalable, high-throughput HE implementation pathway for industrial-grade privacy-preserving federated learning.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Security and Privacy: Applications of cryptography
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Privacy-preserving machine learning is one class of cryptographic methods that aim to analyze private and sensitive data while keeping privacy, such as homomorphic logistic regression training over large encrypted data. In this paper, we propose an efficient algorithm for logistic regression training on large encrypted data using Homomorphic Encryption (HE), which is the mini-batch version of recent methods using a faster gradient variant called $ exttt{quadratic gradient}$. It is claimed that $ exttt{quadratic gradient}$ can integrate curve information (Hessian matrix) into the gradient and therefore can effectively accelerate the first-order gradient (descent) algorithms. We also implement the full-batch version of their method when the encrypted dataset is so large that it has to be encrypted in the mini-batch manner. We compare our mini-batch algorithm with our full-batch implementation method on real financial data consisting of 422,108 samples with 200 freatures. %Our experiments show that Nesterov's accelerated gradient (NAG) Given the inefficiency of HEs, our results are inspiring and demonstrate that the logistic regression training on large encrypted dataset is of practical feasibility, marking a significant milestone in our understanding.

Problem

• Efficient logistic regression training on large encrypted data
• Privacy-preserving machine learning using Homomorphic Encryption
• Accelerating gradient descent with quadratic gradient method

Innovation

• Homomorphic Encryption for logistic regression
• Mini-batch with quadratic gradient acceleration
• Full-batch implementation for large datasets