PAPER: Privacy-Preserving ResNet Models using Low-Degree Polynomial Approximations and Structural Optimizations on Leveled FHE

2025-09-26

AI Summary

Existing FHE-based privacy-preserving inference methods rely on costly cross-layer bootstrapping and high-degree polynomial approximations of activation functions, incurring 2–5% accuracy degradation and excessive multiplicative depth. To address this, we propose a bootstrapping-free FHE inference framework tailored for ResNet architectures. Our method employs a theoretically optimal quadratic ReLU approximation to minimize multiplicative depth; integrates node fusion, weight redistribution, and tower-structured coefficient reuse to reduce circuit depth; and jointly applies parameter clustering and data encoding to recover accuracy. Evaluated on CIFAR-10 and CIFAR-100, our approach achieves up to 4× speedup for ResNet-18/20/32 while matching the accuracy of plaintext ReLU models. This is the first work to simultaneously achieve high efficiency and high fidelity in leveled FHE inference—eliminating bootstrapping entirely while preserving model accuracy and scalability.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent work has made non-interactive privacy-preserving inference more practical by running deep Convolution Neural Network (CNN) with Fully Homomorphic Encryption (FHE). However, these methods remain limited by their reliance on bootstrapping, a costly FHE operation applied across multiple layers, severely slowing inference. They also depend on high-degree polynomial approximations of non-linear activations, which increase multiplicative depth and reduce accuracy by 2-5% compared to plaintext ReLU models. In this work, we focus on ResNets, a widely adopted benchmark architecture in privacy-preserving inference, and close the accuracy gap between their FHE-based non-interactive models and plaintext counterparts, while also achieving faster inference than existing methods. We use a quadratic polynomial approximation of ReLU, which achieves the theoretical minimum multiplicative depth for non-linear activations, along with a penalty-based training strategy. We further introduce structural optimizations such as node fusing, weight redistribution, and tower reuse. These optimizations reduce the required FHE levels in CNNs by nearly a factor of five compared to prior work, allowing us to run ResNet models under leveled FHE without bootstrapping. To further accelerate inference and recover accuracy typically lost with polynomial approximations, we introduce parameter clustering along with a joint strategy of data encoding layout and ensemble techniques. Experiments with ResNet-18, ResNet-20, and ResNet-32 on CIFAR-10 and CIFAR-100 show that our approach achieves up to 4x faster private inference than prior work with comparable accuracy to plaintext ReLU models.

Problem

• Eliminating costly bootstrapping operations in FHE-based private inference
• Closing accuracy gap between FHE models and plaintext ReLU counterparts
• Reducing multiplicative depth and accelerating private CNN inference

Innovation

• Quadratic polynomial approximation minimizes multiplicative depth
• Structural optimizations reduce FHE levels by five times
• Parameter clustering and encoding techniques accelerate inference