AnyTop: Character Animation Diffusion with Any Topology

2025-02-24

AI Summary

Action generation for arbitrary skeletal topologies has long been hindered by data scarcity and structural irregularity. To address this, we propose the first topology-agnostic diffusion framework: skeletal topology is encoded as attention bias, while textual joint descriptions establish cross-skeleton semantic alignment; a topology-aware Transformer denoising network is designed, integrating latent-space semantic disentanglement with text-enhanced joint representation learning. Our method achieves strong generalization to unseen skeletons using only three samples per topology. It excels in downstream tasks—including joint correspondence, temporal segmentation, and motion editing—demonstrating superior performance over prior approaches. The learned latent space exhibits high interpretability and fine-grained controllability, enabling precise, topology-independent motion synthesis. Overall, our framework significantly advances the generalizability and practical applicability of skeleton-agnostic action generation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Generating motion for arbitrary skeletons is a longstanding challenge in computer graphics, remaining largely unexplored due to the scarcity of diverse datasets and the irregular nature of the data. In this work, we introduce AnyTop, a diffusion model that generates motions for diverse characters with distinct motion dynamics, using only their skeletal structure as input. Our work features a transformer-based denoising network, tailored for arbitrary skeleton learning, integrating topology information into the traditional attention mechanism. Additionally, by incorporating textual joint descriptions into the latent feature representation, AnyTop learns semantic correspondences between joints across diverse skeletons. Our evaluation demonstrates that AnyTop generalizes well, even with as few as three training examples per topology, and can produce motions for unseen skeletons as well. Furthermore, our model's latent space is highly informative, enabling downstream tasks such as joint correspondence, temporal segmentation and motion editing. Our webpage, https://anytop2025.github.io/Anytop-page, includes links to videos and code.

Problem

• Generates motion for diverse character skeletons
• Uses skeletal structure as primary input
• Integrates topology into attention mechanism

Innovation

• Diffusion model for motion generation
• Transformer-based denoising network
• Textual joint descriptions integration