PerFace: Metric Learning in Perceptual Facial Similarity for Enhanced Face Anonymization

2025-09-24

AI Summary

Existing face anonymization methods rely on binary identity classification, which fails to capture fine-grained facial similarity (e.g., “highly similar but distinct identities”), thereby compromising the trade-off between anonymity and naturalness. To address this, we propose a human-perception-driven fine-grained face similarity metric: (1) We construct a large-scale perceptual similarity dataset comprising 6,400 manually annotated triplets; (2) We adopt a metric learning framework that jointly optimizes triplet loss and perceptual similarity labels to improve cross-identity similarity estimation. Experiments demonstrate that our model significantly outperforms state-of-the-art baselines in both facial similarity prediction and attribute classification tasks. By enabling more precise quantification of inter-identity similarity, our approach provides a principled foundation for selecting safer and more natural substitute identities in face-swapping-based anonymization. This work overcomes the fundamental limitation of conventional binary classification paradigms in modeling nuanced facial similarity.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Cognitive Modeling & Cognitive Systems: Analogy
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In response to rising societal awareness of privacy concerns, face anonymization techniques have advanced, including the emergence of face-swapping methods that replace one identity with another. Achieving a balance between anonymity and naturalness in face swapping requires careful selection of identities: overly similar faces compromise anonymity, while dissimilar ones reduce naturalness. Existing models, however, focus on binary identity classification "the same person or not", making it difficult to measure nuanced similarities such as "completely different" versus "highly similar but different." This paper proposes a human-perception-based face similarity metric, creating a dataset of 6,400 triplet annotations and metric learning to predict the similarity. Experimental results demonstrate significant improvements in both face similarity prediction and attribute-based face classification tasks over existing methods.

Problem

• Balancing anonymity and naturalness in face swapping
• Measuring nuanced facial similarities beyond binary classification
• Developing human-perception-based metric for facial similarity

Innovation

• Human-perception-based face similarity metric
• Dataset of 6,400 triplet annotations
• Metric learning for nuanced similarity prediction