FG-Portrait: 3D Flow Guided Editable Portrait Animation

2026-03-24

AI Summary

This work proposes a geometry-driven diffusion-based approach to address distortion and identity leakage in video-driven portrait animation, which commonly arise from insufficient disentanglement between motion and identity. By leveraging a parametric 3D head model, the method computes learning-free 3D optical flow to establish precise cross-frame geometric correspondences, which are then encoded as priors into the diffusion model. Combined with depth-guided sampling to align 2D motion dynamics, the framework enables high-fidelity animation generation while preserving source identity. The approach significantly improves motion transfer consistency and visual quality, and further supports flexible user editing of facial expressions and head poses.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Motion transfer from the driving to the source portrait remains a key challenge in the portrait animation. Current diffusion-based approaches condition only on the driving motion, which fails to capture source-to-driving correspondences and consequently yields suboptimal motion transfer. Although flow estimation provides an alternative, predicting dense correspondences from 2D input is ill-posed and often yields inaccurate animation. We address this problem by introducing 3D flows, a learning-free and geometry-driven motion correspondence directly computed from parametric 3D head models. To integrate this 3D prior into diffusion model, we introduce 3D flow encoding to query potential 3D flows for each target pixel to indicate its displacement back to the source location. To obtain 3D flows aligned with 2D motion changes, we further propose depth-guided sampling to accurately locate the corresponding 3D points for each pixel. Beyond high-fidelity portrait animation, our model further supports user-specified editing of facial expression and head pose. Extensive experiments demonstrate the superiority of our method on consistent driving motion transfer as well as faithful source identity preservation.

Problem

• portrait animation
• motion transfer
• 3D flow
• identity preservation
• correspondence estimation

Innovation

• 3D flow
• portrait animation
• motion transfer
• diffusion model
• depth-guided sampling