PartNerFace: Part-based Neural Radiance Fields for Animatable Facial Avatar Reconstruction

2026-04-15

AI Summary

Existing methods exhibit limited generalization to unseen facial expressions and struggle to capture fine-grained motion details. To address this, this work proposes a neural rendering framework based on a parametric head model that maps observed points to a canonical space via inverse skinning. The approach introduces a partitioned deformation field mechanism, where multiple local MLPs independently model deformations in distinct facial regions, and their outputs are fused through a soft weighting strategy. This design significantly enhances the representation of intricate expression details and improves cross-expression generalization. Quantitative and qualitative evaluations demonstrate that the proposed method consistently outperforms current state-of-the-art approaches.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Learning with Manifolds
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We present PartNerFace, a part-based neural radiance fields approach, for reconstructing animatable facial avatar from monocular RGB videos. Existing solutions either simply condition the implicit network with the morphable model parameters or learn an imaginary canonical radiance field, making them fail to generalize to unseen facial expressions and capture fine-scale motion details. To address these challenges, we first apply inverse skinning based on a parametric head model to map an observed point to the canonical space, and then model fine-scale motions with a part-based deformation field. Our key insight is that the deformation of different facial parts should be modeled differently. Specifically, our part-based deformation field consists of multiple local MLPs to adaptively partition the canonical space into different parts, where the deformation of a 3D point is computed by aggregating the prediction of all local MLPs by a soft-weighting mechanism. Extensive experiments demonstrate that our method generalizes well to unseen expressions and is capable of modeling fine-scale facial motions, outperforming state-of-the-art methods both quantitatively and qualitatively.

Problem

• animatable facial avatar
• neural radiance fields
• facial expression generalization
• fine-scale motion
• monocular video reconstruction

Innovation

• part-based deformation
• neural radiance fields
• animatable avatar
• inverse skinning
• local MLPs