ARMO: Autoregressive Rigging for Multi-Category Objects

2025-03-26

AI Summary

Existing 3D generation methods primarily target static models and struggle to support animation requirements for dynamic articulated objects such as humans, animals, and insects. This work addresses rigging—the foundational task enabling skeletal animation—by introducing OmniRig, the first large-scale, multi-category skinned rigging dataset. We propose ARMO, a unified autoregressive framework that models skeletons as complete graphs and discretizes them into token sequences, enabling end-to-end joint localization and topology prediction. To mitigate error accumulation inherent in conventional regression-based approaches, we integrate a mesh-conditioned implicit diffusion model and a joint autoencoder. Evaluated on OmniRig, ARMO achieves state-of-the-art skeletal prediction performance, demonstrates significantly improved cross-category generalization, and enables robust rigging for arbitrary input poses—including non-standard poses beyond canonical A- or T-poses—without pose preprocessing.

Technology Category

• Intelligent Robots: Embodied AI
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recent advancements in large-scale generative models have significantly improved the quality and diversity of 3D shape generation. However, most existing methods focus primarily on generating static 3D models, overlooking the potentially dynamic nature of certain shapes, such as humanoids, animals, and insects. To address this gap, we focus on rigging, a fundamental task in animation that establishes skeletal structures and skinning for 3D models. In this paper, we introduce OmniRig, the first large-scale rigging dataset, comprising 79,499 meshes with detailed skeleton and skinning information. Unlike traditional benchmarks that rely on predefined standard poses (e.g., A-pose, T-pose), our dataset embraces diverse shape categories, styles, and poses. Leveraging this rich dataset, we propose ARMO, a novel rigging framework that utilizes an autoregressive model to predict both joint positions and connectivity relationships in a unified manner. By treating the skeletal structure as a complete graph and discretizing it into tokens, we encode the joints using an auto-encoder to obtain a latent embedding and an autoregressive model to predict the tokens. A mesh-conditioned latent diffusion model is used to predict the latent embedding for conditional skeleton generation. Our method addresses the limitations of regression-based approaches, which often suffer from error accumulation and suboptimal connectivity estimation. Through extensive experiments on the OmniRig dataset, our approach achieves state-of-the-art performance in skeleton prediction, demonstrating improved generalization across diverse object categories. The code and dataset will be made public for academic use upon acceptance.

Problem

• Generating dynamic 3D models with rigging for animation
• Predicting joint positions and connectivity in skeletal structures
• Overcoming error accumulation in regression-based rigging methods

Innovation

• Autoregressive model predicts joint positions and connectivity
• Mesh-conditioned latent diffusion for conditional skeleton generation
• Large-scale rigging dataset with diverse shapes and poses