AHS: Adaptive Head Synthesis via Synthetic Data Augmentations

2026-04-17

AI Summary

Existing avatar replacement methods are constrained by facial crops captured from limited viewpoints, making it challenging to simultaneously preserve diverse expressions, hairstyles, and seamless integration with the upper body. This work proposes an adaptive head synthesis approach that introduces, for the first time, a head reenactment-based data augmentation mechanism operating directly on full upper-body images, enabling self-supervised training without requiring paired data. By leveraging identity-expression disentangled modeling and synthetic data augmentation, the method supports high-quality head replacement across multiple poses while retaining hairstyles and accessories. Experiments demonstrate that the approach consistently generates identity-preserving, visually coherent, and detail-faithful results even under extreme expressions and large pose variations, significantly outperforming current state-of-the-art methods.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Recent digital media advancements have created increasing demands for sophisticated portrait manipulation techniques, particularly head swapping, where one's head is seamlessly integrated with another's body. However, current approaches predominantly rely on face-centered cropped data with limited view angles, significantly restricting their real-world applicability. They struggle with diverse head expressions, varying hairstyles, and natural blending beyond facial regions. To address these limitations, we propose Adaptive Head Synthesis (AHS), which effectively handles full upper-body images with varied head poses and expressions. AHS incorporates a novel head reenacted synthetic data augmentation strategy to overcome self-supervised training constraints, enhancing generalization across diverse facial expressions and orientations without requiring paired training data. Comprehensive experiments demonstrate that AHS achieves superior performance in challenging real-world scenarios, producing visually coherent results that preserve identity and expression fidelity across various head orientations and hairstyles. Notably, AHS shows exceptional robustness in maintaining facial identity while drastic expression changes and faithfully preserving accessories while significant head pose variations.

Problem

• head swapping
• portrait manipulation
• synthetic data augmentation
• head pose variation
• facial expression diversity

Innovation

• Adaptive Head Synthesis
• synthetic data augmentation
• head swapping
• self-supervised learning
• pose-invariant identity preservation