Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers

2026-07-06

AI Summary

This work addresses the performance bottleneck in encrypted Transformer inference caused by excessive computational depth under homomorphic encryption due to sequential nonlinear operations. The authors propose Structured Newton Layer Parallelism (SNLP), the first approach to introduce layer-wise parallelism into encrypted inference, which replaces layer-by-layer nonlinear computations with a small number of solver iterations and linear corrections, substantially reducing circuit depth. By integrating Chebyshev polynomial approximations, the method efficiently implements operations such as softmax and RMSNorm under the CKKS scheme. Evaluated on a 0.5B-parameter model, SNLP reduces the number of ciphertext bootstrappings from 53 to 20—a 2.65× reduction—while increasing perplexity by only 1.2%, and consistently exhibits lower error amplification compared to conventional serial approaches.

Abstract

Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks. We study whether Structured Newton Layer Parallelism (SNLP) can make this inter-layer composition more FHE-friendly: each Transformer block still requires polynomial approximations for operations such as softmax and RMSNorm, but SNLP reduces the layerwise sequential nonlinear depth from L stages to a small number of solver iterations plus linear structured corrections. Using a simulation framework based on Chebyshev polynomial approximations, we measure error accumulation under sequential versus SNLP inference across 8 models and 4 architecture families. On a 0.5B IDN-trained model, SNLP reduces symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation, while lowering error amplification (1.36x vs. 1.42x). Across all tested models, SNLP has lower amplification than sequential inference. Ablations show that softmax approximation dominates the error budget and CKKS arithmetic noise is negligible in our setting, suggesting that SNLP is complementary to block-level FHE-friendly operator design rather than a replacement for it.

Problem

• Fully Homomorphic Encryption
• Transformer inference
• nonlinear depth
• encrypted computation
• sequential composition

Innovation

• Layer-Parallel Inference
• Fully Homomorphic Encryption
• Structured Newton Method
• Transformer Optimization
• Polynomial Approximation