HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion

2026-02-11

AI Summary

Existing methods achieve strong performance in human pose control but struggle to model fine-grained hair dynamics, often resulting in stiff and unrealistic hair motion in animations. This work proposes HairWeaver, the first approach to enable fine-grained control of hair dynamics under a few-shot setting. Built upon a video diffusion model, HairWeaver introduces two lightweight LoRA modules: Motion-Context-LoRA injects motion context information, while Sim2Real-Domain-LoRA enforces appearance consistency through a simulation-to-real domain guidance mechanism. Trained on a CG-simulated dynamic hair dataset, the method achieves state-of-the-art performance across multiple metrics, generating hair animations with high visual realism and natural dynamic details that significantly outperform existing approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We present HairWeaver, a diffusion-based pipeline that animates a single human image with realistic and expressive hair dynamics. While existing methods successfully control body pose, they lack specific control over hair, and as a result, fail to capture the intricate hair motions, resulting in stiff and unrealistic animations. HairWeaver overcomes this limitation using two specialized modules: a Motion-Context-LoRA to integrate motion conditions and a Sim2Real-Domain-LoRA to preserve the subject's photoreal appearance across different data domains. These lightweight components are designed to guide a video diffusion backbone while maintaining its core generative capabilities. By training on a specialized dataset of dynamic human motion generated from a CG simulator, HairWeaver affords fine control over hair motion and ultimately learns to produce highly realistic hair that responds naturally to movement. Comprehensive evaluations demonstrate that our approach sets a new state of the art, producing lifelike human hair animations with dynamic details.

Problem

• hair motion synthesis
• photorealistic animation
• few-shot generation
• video diffusion
• sim-to-real

Innovation

• Hair motion synthesis
• Video diffusion
• LoRA
• Sim-to-real transfer
• Few-shot animation