SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation

2026-07-09

AI Summary

This work addresses the reliance on per-frame skeletal annotations in text-driven 3D mesh animation by proposing a weakly supervised feedforward framework. The method recovers pseudo-skeletal sequences from mesh animations via differentiable skeleton fitting, generates skeleton motions conditioned on input text, and employs Motion-GRPO for temporal optimization to ensure physical plausibility and joint coherence. Without requiring ground-truth skeletal labels, the approach produces explicit, editable 4D animations compatible with industrial pipelines. Evaluated on the Truebones Zoo and Diffusion4D benchmarks, it achieves high-quality text-driven animation across diverse object categories, matching or even surpassing the performance of fully supervised methods.

Abstract

We study 4D generation to synthesize temporally coherent sequences of 3D geometry for animation and content creation. In contrast to existing SDS-based optimization methods and video-driven animation approaches, we adopt a skeleton-driven animation framework aligned with standard industrial pipelines, which enables explicit control and editing. To this end, we propose SkelGen4D, a weakly supervised feed-forward framework for text-driven mesh animation that generates explicit skeleton motions without requiring per-frame skeleton annotations. SkelGen4D first recovers temporally consistent pseudo-skeletons from animated meshes via differentiable fitting, and then generates text-conditioned skeleton motion sequences in a feed-forward manner, further refined with Motion-GRPO to ensure temporally coherent, physically plausible, and articulated animation. We evaluate our method on two large-scale benchmarks, Truebones Zoo and Diffusion4D. Our results show that our weakly supervised skeleton modeling matches or surpasses fully supervised baselines while scaling to diverse object categories for high-quality text-driven mesh animation. Further, our method supports flexible motion editing and is aligned with standard animation production pipelines.

Problem

• 4D generation
• skeleton-based animation
• text-driven mesh animation
• weakly-supervised learning
• temporal coherence

Innovation

• weakly-supervised
• skeleton-based animation
• text-driven 4D generation
• feed-forward framework
• Motion-GRPO