VIGFace: Virtual Identity Generation for Privacy-Free Face Recognition

2024-03-13

AI Summary

To address privacy leakage, copyright risks, and high annotation costs associated with collecting real-face data in face recognition, this paper proposes a novel synthetic face generation paradigm that requires no authentic identity images. Methodologically, it predefines orthogonal virtual identity prototypes in the feature space to guarantee strict separability between synthetic and real individuals; integrates metric learning to construct an identity-aware feature space; and conditions a diffusion model on both virtual and real prototypes to generate high-fidelity, identity-consistent faces. Experiments demonstrate that the method surpasses state-of-the-art approaches across multiple benchmarks. Critically, models trained solely on synthetic data achieve significantly improved generalization performance, while completely eliminating privacy leakage risks inherent in real-data collection.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Privacy
• Natural Language Processing: Generation

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Deep learning-based face recognition continues to face challenges due to its reliance on huge datasets obtained from web crawling, which can be costly to gather and raise significant real-world privacy concerns. To address this issue, we propose VIGFace, a novel framework capable of generating synthetic facial images. Our idea originates from pre-assigning virtual identities in the feature space. Initially, we train the face recognition model using a real face dataset and create a feature space for both real and virtual identities, where virtual prototypes are orthogonal to other prototypes. Subsequently, we train the diffusion model based on the established feature space, enabling it to generate authentic human face images from real prototypes and synthesize virtual face images from virtual prototypes. Our proposed framework provides two significant benefits. Firstly, it shows clear separability between existing individuals and virtual face images, allowing one to create synthetic images with confidence and without concerns about privacy and portrait rights. Secondly, it ensures improved performance through data augmentation by incorporating real existing images. Extensive experiments demonstrate the superiority of our virtual face dataset and framework, outperforming the previous state-of-the-art on various face recognition benchmarks.

Problem

• Addresses privacy concerns in face recognition datasets.
• Generates synthetic facial images using virtual identities.
• Enhances face recognition performance via data augmentation.

Innovation

• Generates synthetic facial images using virtual identities
• Uses orthogonal virtual prototypes in feature space
• Trains diffusion model for authentic and virtual face synthesis