Benchmarking Face Recognition without Real Faces

2026-07-16

AI Summary

This study addresses the privacy and ethical challenges inherent in facial recognition evaluation by systematically assessing the viability of synthetic face datasets as alternatives to real human data. Evaluating twelve synthetic datasets across verification metrics, similarity distributions, model ranking consistency, and data distribution characteristics, the work conducts a comprehensive comparison using twenty-four pre-trained models—spanning both CNN and Transformer architectures—on seven real and twelve synthetic benchmarks. The findings demonstrate for the first time that high-quality synthetic datasets such as MorphFace and Vec2Face achieve evaluation reliability comparable to real-world benchmarks, with model behavior reproducibility matching the natural variation observed between real datasets. These results establish a feasible pathway toward fully privacy-preserving training and evaluation pipelines in facial recognition research.

Abstract

Synthetic face datasets have become effective enough to train face recognition models with accuracy rivaling that of models trained on real photographs. This progress sidesteps the ethical and legal burdens of collecting real biometric data, yet evaluation has not kept pace. Even studies that train entirely on synthetic images still rely on real-face benchmarks to measure performance, leaving the privacy problem only half solved. We ask whether synthetic datasets can replace real benchmarks for face recognition evaluation. We test 12 synthetic datasets against 7 established real benchmarks using 24 pre-trained models that span both convolutional and transformer architectures. Our evaluation covers biometric verification metrics, similarity score distributions, cross-model ranking consistency, and the underlying distributional properties of each dataset. Benchmarking fidelity varies widely across the synthetic candidates, but the two strongest, MorphFace and Vec2Face, reproduce the relative behavior of real benchmarks and reach agreement levels that fall within the natural disagreement already observed among the real benchmarks themselves. These results establish that well-constructed synthetic datasets can support reliable comparative evaluation for face recognition, moving the field closer to a fully synthetic and privacy-preserving pipeline for both training and benchmarking.

Problem

• face recognition
• synthetic datasets
• benchmarking
• privacy
• biometric evaluation

Innovation

• synthetic face datasets
• face recognition benchmarking
• privacy-preserving evaluation
• distributional fidelity
• model ranking consistency