From Synthetic to Real: Toward Identity-Consistent Makeup Transfer with Synthetic and Real Data

2026-05-08

AI Summary

Existing makeup transfer methods suffer significant performance degradation in complex real-world scenarios due to insufficient identity consistency and the domain gap between synthetic and real data. To address this, this work introduces ConsistentBeauty, a data generation pipeline that produces synthetic images with high fidelity in both identity preservation and makeup appearance. Building upon this, we propose RealBeauty, a novel framework that integrates diffusion models, synthetic-data supervision, and reinforcement learning with a verifiable reward mechanism to enable effective synthetic-to-real adaptive transfer. Our approach achieves state-of-the-art performance across multiple benchmarks, substantially improving identity retention and makeup transfer quality in real-world settings. Additionally, we release a new evaluation benchmark encompassing diverse human demographics to facilitate future research.

Abstract

Makeup transfer aims to apply the makeup style of a reference portrait to a source portrait while preserving identity and background. Early methods formulate this task as unsupervised image-to-image translation, relying on surrogate objectives and often yielding limited performance. Recent diffusion- and flow-based approaches instead exploit synthetic data for supervised training, leading to significant improvements. However, these methods still face two critical challenges: synthetic supervision frequently fails to faithfully preserve identity, and the domain gap between synthetic and real data limits generalization, resulting in degraded performance in complex real-world scenarios. To address these issues, this paper first proposes ConsistentBeauty, a novel data curation pipeline that ensures makeup fidelity and strict identity consistency within the synthesized data. Second, we propose RealBeauty, a synthetic-to-real post-training framework. Beyond supervised learning on curated synthetic data, we further adapt the model to real-world scenarios through reinforcement learning and design novel verifiable rewards tailored to the makeup transfer task. It allows the model to further benefit from real makeup patterns beyond synthetic supervision. In addition, we establish a new diverse benchmark for makeup transfer, covering a wide range of skin tones, ages, genders, poses, and makeup styles, thereby enabling a more comprehensive evaluation of model performance under diverse real-world conditions. Extensive experiments show that our method achieves state-of-the-art performance on multiple benchmarks and demonstrates clear advantages in identity preservation and performance on complex real-world cases.

Problem

• makeup transfer
• identity consistency
• synthetic-to-real domain gap
• generalization

Innovation

• makeup transfer
• identity consistency
• synthetic-to-real adaptation
• reinforcement learning
• data curation