FaceRefiner: High-Fidelity Facial Texture Refinement With Differentiable Rendering-Based Style Transfer
Existing facial texture generation methods suffer from limited generalization on in-the-wild images, often producing UV textures that deviate from the input in terms of fine details, structural fidelity, and identity consistency. To address this, this work proposes FaceRefiner, which introduces differentiable rendering into a style transfer framework for the first time. By treating 3D-sampled textures as style and generated textures as content, FaceRefiner enables pixel-wise, multi-level (low-, mid-, and high-level) information transfer directly in UV space. This approach significantly enhances both photorealism and identity preservation in the synthesized textures. Extensive experiments on Multi-PIE, CelebA, and FFHQ demonstrate that FaceRefiner consistently outperforms state-of-the-art methods by a substantial margin.