CryptoFace: End-to-End Encrypted Face Recognition

2025-08-29

AI Summary

Facial recognition poses severe privacy risks due to potential biometric data leakage. To address this, we propose the first end-to-end fully homomorphic encryption (FHE)-based facial recognition system, covering feature extraction, encrypted storage, and similarity matching—ensuring both raw images and feature vectors remain encrypted throughout the entire pipeline. Our method introduces a novel shallow Patch-convolutional hybrid network architecture, enabling efficient parallel FHE evaluation over high-dimensional tensors, and incorporates a block-wise tensor processing mechanism that achieves resolution-agnostic, near-constant inference latency while substantially reducing computational overhead. Evaluated on standard face verification benchmarks, our system achieves significantly faster inference than prior FHE-based neural networks and attains higher recognition accuracy. This work constitutes the first provably secure, end-to-end encrypted facial recognition framework, delivering a practical solution for privacy-critical applications.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Face recognition is central to many authentication, security, and personalized applications. Yet, it suffers from significant privacy risks, particularly arising from unauthorized access to sensitive biometric data. This paper introduces CryptoFace, the first end-to-end encrypted face recognition system with fully homomorphic encryption (FHE). It enables secure processing of facial data across all stages of a face-recognition process--feature extraction, storage, and matching--without exposing raw images or features. We introduce a mixture of shallow patch convolutional networks to support higher-dimensional tensors via patch-based processing while reducing the multiplicative depth and, thus, inference latency. Parallel FHE evaluation of these networks ensures near-resolution-independent latency. On standard face recognition benchmarks, CryptoFace significantly accelerates inference and increases verification accuracy compared to the state-of-the-art FHE neural networks adapted for face recognition. CryptoFace will facilitate secure face recognition systems requiring robust and provable security. The code is available at https://github.com/human-analysis/CryptoFace.

Problem

• Securing face recognition against biometric data privacy risks
• Enabling encrypted processing throughout feature extraction and matching
• Reducing inference latency while maintaining verification accuracy

Innovation

• End-to-end encrypted face recognition system
• Uses fully homomorphic encryption technology
• Patch-based convolutional networks reduce latency