CryptPEFT: Efficient and Private Neural Network Inference via Parameter-Efficient Fine-Tuning

2025-08-17

AI Summary

To address the dual privacy challenge—protecting both user inputs and fine-tuned adapters during inference in parameter-efficient fine-tuning (PEFT) scenarios—this paper proposes CryptPEFT. Methodologically, it introduces the first single-round-communication privacy-preserving inference framework tailored for PEFT, strictly confining encrypted computation to lightweight adapters. It synergistically integrates secure multi-party computation (MPC) with automated neural architecture search (NAS) to jointly optimize adapter topology and cryptographic protocol design. Evaluated on Vision Transformers (ViT), CryptPEFT achieves up to 291.5× faster inference than baseline privacy-preserving methods, attains 85.47% accuracy on CIFAR-100, and incurs only 2.26 seconds of end-to-end latency—demonstrating a significant advance in simultaneously balancing security, efficiency, and model utility.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Security and Privacy: Applications of cryptography
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Publicly available large pretrained models (i.e., backbones) and lightweight adapters for parameter-efficient fine-tuning (PEFT) have become standard components in modern machine learning pipelines. However, preserving the privacy of both user inputs and fine-tuned adapters -- often trained on sensitive data -- during inference remains a significant challenge. Applying cryptographic techniques, such as multi-party computation (MPC), to PEFT settings still incurs substantial encrypted computation across both the backbone and adapter, mainly due to the inherent two-way communication between them. To address this limitation, we propose CryptPEFT, the first PEFT solution specifically designed for private inference scenarios. CryptPEFT introduces a novel one-way communication (OWC) architecture that confines encrypted computation solely to the adapter, significantly reducing both computational and communication overhead. To maintain strong model utility under this constraint, we explore the design space of OWC-compatible adapters and employ an automated architecture search algorithm to optimize the trade-off between private inference efficiency and model utility. We evaluated CryptPEFT using Vision Transformer backbones across widely used image classification datasets. Our results show that CryptPEFT significantly outperforms existing baselines, delivering speedups ranging from $20.62 imes$ to $291.48 imes$ in simulated wide-area network (WAN) and local-area network (LAN) settings. On CIFAR-100, CryptPEFT attains 85.47% accuracy with just 2.26 seconds of inference latency. These findings demonstrate that CryptPEFT offers an efficient and privacy-preserving solution for modern PEFT-based inference.

Problem

• Privacy preservation in neural network inference
• Reducing encrypted computation in PEFT settings
• Optimizing trade-off between efficiency and model utility

Innovation

• One-way communication architecture for privacy
• Automated search for OWC-compatible adapters
• Confines encrypted computation to adapter only