Reverse Personalization

2025-12-28

AI Summary

Existing face anonymization methods rely on pretrained identity representations or require model fine-tuning, struggling to simultaneously achieve attribute controllability and cross-identity generalization. This paper proposes a reverse personalization framework that enables fine-grained de-identification for unseen identities directly via the inverse process of conditional diffusion models—without text prompts or model adaptation. Our core contributions are: (1) the first formulation of reverse personalization as a principled paradigm; (2) an identity-guided dual-branch conditioning mechanism; and (3) latent-space attribute disentanglement coupled with editable attribute modulation, enabling explicit attribute preservation and control. Evaluated on identity removal, attribute fidelity, and image quality, our method achieves state-of-the-art trade-offs across all three dimensions, significantly enhancing both controllability and generalizability of face anonymization. Code and dataset are publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Privacy
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent text-to-image diffusion models have demonstrated remarkable generation of realistic facial images conditioned on textual prompts and human identities, enabling creating personalized facial imagery. However, existing prompt-based methods for removing or modifying identity-specific features rely either on the subject being well-represented in the pre-trained model or require model fine-tuning for specific identities. In this work, we analyze the identity generation process and introduce a reverse personalization framework for face anonymization. Our approach leverages conditional diffusion inversion, allowing direct manipulation of images without using text prompts. To generalize beyond subjects in the model's training data, we incorporate an identity-guided conditioning branch. Unlike prior anonymization methods, which lack control over facial attributes, our framework supports attribute-controllable anonymization. We demonstrate that our method achieves a state-of-the-art balance between identity removal, attribute preservation, and image quality. Source code and data are available at https://github.com/hanweikung/reverse-personalization .

Problem

• Develops a reverse personalization framework for face anonymization
• Enables identity removal without text prompts or model fine-tuning
• Achieves attribute-controllable anonymization with high image quality

Innovation

• Reverse personalization framework for face anonymization
• Conditional diffusion inversion without text prompts
• Identity-guided conditioning for attribute-controllable anonymization