Towards Zero Rotation and Beyond: Architecting Neural Networks for Fast Secure Inference with Homomorphic Encryption

2026-01-29

AI Summary

Neural network inference under homomorphic encryption (HE) suffers from significant computational overhead, as most existing models are designed for plaintext and incur severe performance bottlenecks when naively adapted to HE. To address this, this work proposes StriaNet, the first neural architecture natively optimized for HE. StriaNet introduces the StriaBlock module guided by two key design principles: focus constraints and channel-packing-aware scaling. It incorporates several HE-specific optimizations—including ExRot-Free convolution, Cross Kernel, rotation-efficient operations, and adaptive bottleneck ratios—to drastically reduce both external and internal rotations. Evaluated on ImageNet, Tiny ImageNet, and CIFAR-10, StriaNet achieves 9.78×, 6.01×, and 9.24× inference speedups, respectively, while maintaining comparable accuracy to conventional models.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Heuristic Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Privacy-enhancing technologies
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Privacy-preserving deep learning addresses privacy concerns in Machine Learning as a Service (MLaaS) by using Homomorphic Encryption (HE) for linear computations. However, the computational overhead remains a major challenge. While prior work has improved efficiency, most approaches build on models originally designed for plaintext inference. Such models incur architectural inefficiencies when adapted to HE. We argue that substantial gains require networks tailored to HE rather than retrofitting plaintext architectures. Our design has two components: the building block and the overall architecture. First, StriaBlock targets the most expensive HE operation, rotation. It integrates ExRot-Free Convolution and a novel Cross Kernel, eliminating external rotations and requiring only 19% of the internal rotations used by plaintext models. Second, our architectural principles include (i) the Focused Constraint Principle, which limits cost-sensitive factors while preserving flexibility elsewhere, and (ii) the Channel Packing-Aware Scaling Principle, which adapts bottleneck ratios to ciphertext channel capacity that varies with depth. Together, these strategies control both local and end-to-end HE cost, enabling a balanced HE-tailored network. We evaluate the resulting StriaNet across datasets of varying scales, including ImageNet, Tiny ImageNet, and CIFAR-10. At comparable accuracy, StriaNet achieves speedups of 9.78x, 6.01x, and 9.24x on ImageNet, Tiny ImageNet, and CIFAR-10, respectively.

Problem

• Homomorphic Encryption
• Secure Inference
• Neural Network Architecture
• Rotation Operation
• Privacy-Preserving Machine Learning

Innovation

• Homomorphic Encryption
• Rotation-Free Architecture
• StriaBlock
• Channel Packing
• Privacy-Preserving Inference