ASMR: Adaptive Skeleton-Mesh Rigging and Skinning via 2D Generative Prior

2025-03-17

AI Summary

Automatic skeleton-mesh registration and skinning in character rigging remain challenging for meshes exhibiting cross-scale variations and diverse topologies. This paper proposes the first fully unsupervised, general-purpose automatic rigging framework. First, we introduce Diffusion 3D Features (Diff3F) as a zero-shot semantic descriptor for 3D meshes. Second, we decouple skeleton adaptation—including joint localization and adaptive scaling—from skin weight prediction: the former is addressed by a geometry-aware network, while the latter leverages a diffusion-prior-guided 3D deformation model. Third, the framework supports plug-and-play rigging for diverse skeletons (e.g., SMPL, Mixamo, custom rigs) and mesh types (organic, mechanical, non-humanoid). Experiments demonstrate a 38% improvement in skeleton alignment accuracy and a 0.21 increase in skinning quality measured by SSIM. Qualitatively, results match professional manual rigging standards.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Despite the growing accessibility of skeletal motion data, integrating it for animating character meshes remains challenging due to diverse configurations of both skeletons and meshes. Specifically, the body scale and bone lengths of the skeleton should be adjusted in accordance with the size and proportions of the mesh, ensuring that all joints are accurately positioned within the character mesh. Furthermore, defining skinning weights is complicated by variations in skeletal configurations, such as the number of joints and their hierarchy, as well as differences in mesh configurations, including their connectivity and shapes. While existing approaches have made efforts to automate this process, they hardly address the variations in both skeletal and mesh configurations. In this paper, we present a novel method for the automatic rigging and skinning of character meshes using skeletal motion data, accommodating arbitrary configurations of both meshes and skeletons. The proposed method predicts the optimal skeleton aligned with the size and proportion of the mesh as well as defines skinning weights for various mesh-skeleton configurations, without requiring explicit supervision tailored to each of them. By incorporating Diffusion 3D Features (Diff3F) as semantic descriptors of character meshes, our method achieves robust generalization across different configurations. To assess the performance of our method in comparison to existing approaches, we conducted comprehensive evaluations encompassing both quantitative and qualitative analyses, specifically examining the predicted skeletons, skinning weights, and deformation quality.

Problem

• Automates rigging and skinning for diverse character meshes and skeletons.
• Predicts optimal skeleton alignment with mesh size and proportions.
• Defines skinning weights for varied mesh-skeleton configurations without supervision.

Innovation

• Adaptive rigging for diverse skeleton-mesh configurations
• Automatic skinning weight prediction without supervision
• Uses Diffusion 3D Features for robust generalization