Machine Pareidolia: Protecting Facial Image with Emotional Editing

2026-03-03

AI Summary

Existing black-box facial privacy protection methods suffer from poor transferability and limited generalization across diverse demographics, particularly with respect to gender and skin tone. This work proposes a novel approach that, for the first time, integrates facial expression editing into identity obfuscation through a dual-objective optimization framework. By jointly fine-tuning a scoring network to simultaneously control target identity and expression, and incorporating gradient projection with local smoothness regularization, the method ensures convergence to a shared local optimum. It demonstrates superior adaptability under unconventional imaging conditions, outperforming baseline techniques—including noise injection, makeup-based perturbations, and free-attribute manipulation—both qualitatively in visual fidelity and quantitatively across standard metrics, while effectively evading major commercial online face recognition APIs.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Privacy
• Search and Optimization: Local Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Security and Privacy: Large-scale security measurements

Abstract

The proliferation of facial recognition (FR) systems has raised privacy concerns in the digital realm, as malicious uses of FR models pose a significant threat. Traditional countermeasures, such as makeup style transfer, have suffered from low transferability in black-box settings and limited applicability across various demographic groups, including males and individuals with darker skin tones. To address these challenges, we introduce a novel facial privacy protection method, dubbed \textbf{MAP}, a pioneering approach that employs human emotion modifications to disguise original identities as target identities in facial images. Our method uniquely fine-tunes a score network to learn dual objectives, target identity and human expression, which are jointly optimized through gradient projection to ensure convergence at a shared local optimum. Additionally, we enhance the perceptual quality of protected images by applying local smoothness regularization and optimizing the score matching loss within our network. Empirical experiments demonstrate that our innovative approach surpasses previous baselines, including noise-based, makeup-based, and freeform attribute methods, in both qualitative fidelity and quantitative metrics. Furthermore, MAP proves its effectiveness against an online FR API and shows advanced adaptability in uncommon photographic scenarios.

Problem

• facial recognition
• privacy protection
• machine pareidolia
• black-box setting
• demographic bias

Innovation

• facial privacy protection
• emotion-based identity disguise
• score network fine-tuning
• gradient projection optimization
• perceptual quality enhancement