Prot'eg'e: Learn and Generate Basic Makeup Styles with Generative Adversarial Networks (GANs)

2024-12-29

AI Summary

Existing digital makeup systems rely either on manual design or predefined templates, limiting their ability to automatically generate diverse, high-fidelity base makeup tailored to individual facial characteristics. This paper proposes the first end-to-end conditional generative adversarial network (cGAN)-based framework for personalized makeup generation. Our approach innovatively integrates facial semantic segmentation with style-decoupled representation learning, enabling controllable disentanglement of makeup components (e.g., eyeshadow, lip color) and cross-face adaptive synthesis. Unlike conventional recommendation- or transfer-based methods, ours requires no user expertise or interactive intervention, substantially lowering usability barriers. User studies demonstrate significant improvements in both ease of use and creative satisfaction. Furthermore, the framework supports real-time inference on mobile devices.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Makeup is no longer confined to physical application; people now use mobile apps to digitally apply makeup to their photos, which they then share on social media. However, while this shift has made makeup more accessible, designing diverse makeup styles tailored to individual faces remains a challenge. This challenge currently must still be done manually by humans. Existing systems, such as makeup recommendation engines and makeup transfer techniques, offer limitations in creating innovative makeups for different individuals"intuitively"-- significant user effort and knowledge needed and limited makeup options available in app. Our motivation is to address this challenge by proposing Prot'eg'e, a new makeup application, leveraging recent generative model -- GANs to learn and automatically generate makeup styles. This is a task that existing makeup applications (i.e., makeup recommendation systems using expert system and makeup transfer methods) are unable to perform. Extensive experiments has been conducted to demonstrate the capability of Prot'eg'e in learning and creating diverse makeups, providing a convenient and intuitive way, marking a significant leap in digital makeup technology!

Problem

• Diverse Makeup Styles
• Personalized Facial Features
• Limited Cosmetic Options

Innovation

• Generative Adversarial Networks
• Personalized Makeup Generation
• Digital Makeup Technology