MakeupMirror: Improving Facial Attribute Preservation in Diffusion Models for Makeup Transfer

2026-06-18

AI Summary

Existing diffusion-based makeup transfer methods suffer from inadequate identity preservation and skin tone fidelity, limiting the realism and practicality of virtual try-on applications. To address these limitations, this work proposes MakeupMirror, a high-fidelity makeup transfer approach that integrates facial geometric guidance via ControlNet, region-aware precise makeup application, a skin tone consistency modulation mechanism, and an efficient Levenberg-Marquardt Langevin sampling strategy. This framework achieves natural-looking makeup transfer while faithfully preserving identity characteristics. Experimental results demonstrate that, compared to Stable-Makeup, MakeupMirror improves face similarity by 60%, reduces skin tone discrepancy by 50%, and requires only 0.7 seconds for inference. Notably, expert evaluations report a 94% acceptance rate regarding identity preservation.

Abstract

Makeup transfer models enable fun augmented reality (AR) experiences as well as virtual try-on (VTO) for online makeup shopping. While recent state-of-the-art diffusion based solutions such as Stable-Makeup dramatically improve the accuracy and realism of makeup transfer, they still face limitations in identity and skin color preservation, making production-level VTO for makeup shopping unrealistic. In this work, we propose MakeupMirror, a diffusion-based approach to makeup transfer that makes significant progress towards preserving facial features and skin tone. We introduce several technical innovations over Stable-Makeup: (1) integration of facial geometry conditioning with ControlNets to maintain facial fidelity; (2) region-specific makeup transfer control to enable precise makeup application across facial regions such as skin, eyes and lips; (3) skin tone-based makeup transfer modulation that prevent skin tone alteration in cross-subject transfer scenarios; and (4) integration of a Levenberg-Marquardt Langevin sampler to speed up inference while maintaining generation quality. Our experiments on CPM-Real, Makeup Wild, and (herein newly collected, more diverse) MakeupSelfies datasets show that MakeupMirror improves relative facial recognition similarity by +60%, reduces relative skin tone difference by -50% over Stable-Makeup, with a latency of 0.7s, while achieving expert acceptance rate of 94% across core facial identity preservation criteria.

Problem

• makeup transfer
• facial attribute preservation
• identity preservation
• skin tone preservation
• diffusion models

Innovation

• diffusion models
• makeup transfer
• facial attribute preservation
• ControlNet
• skin tone modulation