RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data

2025-11-23

AI Summary

Facial mesh topologies exhibit high variability—including multiple disconnected components—and high-quality annotations for FACS-based blendshape rigging are scarce. Method: We propose RigAnyFace, a scalable neural auto-rigging framework that employs a triangulation-agnostic surface learning network and introduces a novel 2D weak-supervision strategy, eliminating the need for 3D ground-truth labels. The architecture is specifically designed to model FACS parameters and jointly deform multi-component structures (e.g., eyeballs). Contribution/Results: RigAnyFace achieves state-of-the-art performance on both artist-created assets and real-world facial capture data. It is the first method to enable fine-grained, FACS-compliant rigging of meshes with multiple disconnected components. Moreover, it demonstrates unprecedented generalization across diverse topologies and significantly improves binding accuracy compared to prior approaches.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Learning with Manifolds
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In this paper, we present RigAnyFace (RAF), a scalable neural auto-rigging framework for facial meshes of diverse topologies, including those with multiple disconnected components. RAF deforms a static neutral facial mesh into industry-standard FACS poses to form an expressive blendshape rig. Deformations are predicted by a triangulation-agnostic surface learning network augmented with our tailored architecture design to condition on FACS parameters and efficiently process disconnected components. For training, we curated a dataset of facial meshes, with a subset meticulously rigged by professional artists to serve as accurate 3D ground truth for deformation supervision. Due to the high cost of manual rigging, this subset is limited in size, constraining the generalization ability of models trained exclusively on it. To address this, we design a 2D supervision strategy for unlabeled neutral meshes without rigs. This strategy increases data diversity and allows for scaled training, thereby enhancing the generalization ability of models trained on this augmented data. Extensive experiments demonstrate that RAF is able to rig meshes of diverse topologies on not only our artist-crafted assets but also in-the-wild samples, outperforming previous works in accuracy and generalizability. Moreover, our method advances beyond prior work by supporting multiple disconnected components, such as eyeballs, for more detailed expression animation. Project page: https://wenchao-m.github.io/RigAnyFace.github.io

Problem

• Automating facial mesh rigging for diverse topologies using neural networks
• Overcoming limited labeled data with 2D supervision for unlabeled meshes
• Supporting disconnected components like eyeballs for detailed expression animation

Innovation

• Neural auto-rigging framework for diverse facial topologies
• Triangulation-agnostic surface network processing disconnected components
• 2D supervision strategy scaling training with unlabeled data