H-Adapter: Pose-Robust Hairstyle Transfer via Attention-Derived, Source-Aligned Hair Masks

2026-06-24

AI Summary

This work addresses the challenge of hairstyle transfer under large head pose discrepancies between source and reference images by proposing a novel diffusion-based approach. The method introduces a spatially disentangled cross-attention mechanism coupled with region-specific loss functions, enabling the model to extract hair editing masks from the reference image that are accurately aligned with the source image. This alignment facilitates precise guidance of the hairstyle transfer process, significantly enhancing robustness under substantial pose variations. The proposed framework effectively preserves identity features and non-hair regions while faithfully reconstructing fine-grained details of the reference hairstyle. Experimental results demonstrate state-of-the-art performance on pose-diverse subsets, achieving the best scores in FID, FID_CLIP, and CLIP-I metrics. Furthermore, the approach supports extended functionalities such as text-guided generation and hair color control.

Abstract

Hairstyle transfer has practical applications such as virtual try-on, yet remains challenging when the source and reference exhibit large head-pose discrepancies. We propose H-Adapter, which improves pose robustness by training with a region-specific loss that disentangles hair and non-hair objectives and thereby induces spatially disentangled cross-attention, from which a source-aligned hair edit mask is derived to guide diffusion-based inpainting. Experiments on pose-agnostic and pose-different subsets demonstrate strong quantitative results, including the best FID, $\mathrm{FID}_{\mathrm{CLIP}}$, and CLIP-I under pose differences, while maintaining competitive non-hair preservation and improving qualitative fidelity to fine-grained reference hairstyle details. Beyond source-conditioned transfer, H-Adapter supports practical extensions including text-to-image generation, auxiliary prompt-based hair color control, and compatibility with an identity-preserving IP-Adapter variant. We also introduce a VLM-as-a-judge protocol and observe consistent gains in hairstyle faithfulness, non-hair preservation, and artifact quality.

Problem

• hairstyle transfer
• pose robustness
• head pose discrepancy
• virtual try-on
• diffusion-based inpainting

Innovation

• pose-robust hairstyle transfer
• source-aligned hair mask
• spatially disentangled cross-attention
• diffusion-based inpainting
• region-specific loss