RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assets

2025-02-13

AI Summary

This work addresses the problem of automatic skinning and rigging for diverse 3D models—including non-anthropomorphic assets—by jointly solving joint localization, skeletal topology inference, and skinning weight assignment without requiring predefined templates. The proposed method introduces an end-to-end differentiable framework that unifies these tasks via an autoregressive Transformer architecture coupled with a BFS-ordered skeletal representation, enabling joint probabilistic modeling of joints, bones, and weights. A diffusion-guided refinement module is incorporated to enhance keypoint localization accuracy. The model operates directly on raw 3D geometry and supports arbitrary shape categories, trained end-to-end in a fully supervised manner. Evaluated on RigNet and Objaverse benchmarks, it achieves state-of-the-art performance across humanoids, quadrupeds, marine creatures, insects, and other categories. It demonstrates superior quality, robustness, generalizability, and inference efficiency compared to existing approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present RigAnything, a novel autoregressive transformer-based model, which makes 3D assets rig-ready by probabilistically generating joints, skeleton topologies, and assigning skinning weights in a template-free manner. Unlike most existing auto-rigging methods, which rely on predefined skeleton template and are limited to specific categories like humanoid, RigAnything approaches the rigging problem in an autoregressive manner, iteratively predicting the next joint based on the global input shape and the previous prediction. While autoregressive models are typically used to generate sequential data, RigAnything extends their application to effectively learn and represent skeletons, which are inherently tree structures. To achieve this, we organize the joints in a breadth-first search (BFS) order, enabling the skeleton to be defined as a sequence of 3D locations and the parent index. Furthermore, our model improves the accuracy of position prediction by leveraging diffusion modeling, ensuring precise and consistent placement of joints within the hierarchy. This formulation allows the autoregressive model to efficiently capture both spatial and hierarchical relationships within the skeleton. Trained end-to-end on both RigNet and Objaverse datasets, RigAnything demonstrates state-of-the-art performance across diverse object types, including humanoids, quadrupeds, marine creatures, insects, and many more, surpassing prior methods in quality, robustness, generalizability, and efficiency. Please check our website for more details: https://www.liuisabella.com/RigAnything.

Problem

• Template-free 3D rigging
• Autoregressive joint prediction
• Diverse skeleton generation

Innovation

• Autoregressive transformer-based model
• Template-free joint generation
• Diffusion modeling for accuracy