Lightweight, Practical Encrypted Face Recognition with GPU Support

2026-04-01

AI Summary

This work addresses the privacy risks inherent in facial embedding similarity search under client-server architectures, where existing fully homomorphic encryption (FHE) approaches remain impractical due to excessive computational and memory overhead. To overcome these limitations, the authors propose the BSGS-Diagonal algorithm, which substantially reduces the number of required rotation keys, and introduce a fused GPU kernel design that minimizes data movement through operation fusion and memory optimization. Built upon the CKKS scheme and integrated with FIDESlib/OpenFHE, the resulting system enables end-to-end encrypted deployment on resource-constrained devices, reducing client-side memory usage by approximately 14 GB and maintaining server peak memory below 10 GB. The GPU-accelerated implementation achieves up to 9× and 17× speedups, respectively, enabling sub-second identification in databases of up to 32K embeddings.

Abstract

Face recognition models operate in a client-server setting where a client extracts a compact face embedding and a server performs similarity search over a template database. This raises privacy concerns, as facial data is highly sensitive. To provide cryptographic privacy guarantees, one can use fully homomorphic encryption to perform end-to-end encrypted similarity search. However, existing FHE-based protocols are computationally costly and, impose high memory overhead. Building on prior work, HyDia, we introduce algorithmic and system-level improvements targeting real-world deployment with resource-constrained clients. First, we propose BSGS-Diagonal, an algorithm delivering fast and memory-efficient similarity computation. BSGS-Diagonal substantially shrinks the rotation-key set, lowering both client and server memory requirements, and also improves practical server runtime. This yields a 91% reduction in the number of rotation keys, translating to approximately 14 GB less memory used on the client, and reducing overall CPU peak RAM from over 30 GB in the original HyDia to under 10 GB for databases up to size 1M. In addition, runtime is improved by up to 1.57x for the membership verification scenario and 1.43x for the identification scenario. Secondly, we introduce fully GPU-optimized similarity matrix computation kernels. The implementation is built upon FIDESlib, a CKKS-level GPU library based on OpenFHE. Rather than offloading individual CKKS primitives in isolation, the integrated kernels fuse operations to avoid repeated CPU-GPU ciphertext movement and costly FIDESlib/OpenFHE data-structure conversions. As a result, our GPU implementations of both HyDia and BSGS-Diagonal achieve up to 9x and 17x speedups, respectively, enabling sub-second encrypted face recognition for databases up to 32K entries while further reducing host memory usage.

Problem

• encrypted face recognition
• fully homomorphic encryption
• privacy-preserving
• GPU acceleration
• memory efficiency

Innovation

• fully homomorphic encryption
• encrypted face recognition
• GPU acceleration
• rotation key optimization
• CKKS scheme