Revisiting ML Training under Fully Homomorphic Encryption: Convergence Guarantees, Differential Privacy, and Efficient Algorithms

2026-05-26

AI Summary

This work addresses the lack of theoretical convergence guarantees in existing fully homomorphic encryption (FHE)-based machine learning training and the high computational overhead of conventional differential privacy (DP) approaches, which hinder scalability. We propose an efficient FHE-compatible approximate gradient descent algorithm that integrates DP by employing data-independent polynomial approximations of activation and loss functions. This design circumvents per-sample gradient clipping and enables provably convergent private training directly in the encrypted domain. To the best of our knowledge, this is the first work to establish theoretical convergence guarantees for ML training under FHE. Our method achieves significantly improved computational efficiency while maintaining utility comparable to standard DP-GD, thereby advancing practical, secure, and scalable privacy-preserving machine learning on sensitive data.

Abstract

We present the first theoretical convergence analysis of machine learning training under fully homomorphic encryption (FHE), combined with a differentially private (DP) training algorithm tailored to encrypted computation. Our approach improves computational efficiency over standard differentially private gradient descent (DP-GD) while achieving comparable utility. In particular, we prove convergence of approximate gradient descent using polynomial approximations of activation and loss functions, which are required for FHE compatibility. To preserve privacy in downstream tasks, we integrate differential privacy without relying on costly per-sample gradient clipping, enabling scalable encrypted learning. We also provide data-independent hyperparameter selection and theoretically grounded strategies for polynomial approximation which can be of independent interest. Together, these contributions advance the feasibility of efficient, private, and secure machine learning on sensitive data.

Problem

• Fully Homomorphic Encryption
• Machine Learning Training
• Convergence Guarantees
• Differential Privacy
• Encrypted Computation

Innovation

• Fully Homomorphic Encryption
• Differential Privacy
• Convergence Analysis
• Polynomial Approximation
• Encrypted Machine Learning