LUNA: Learning Universal 3D Human Animation Beyond Skinning

2026-06-30

AI Summary

This work addresses the limited expressiveness and fitting artifacts of existing linear blend skinning (LBS)-based parametric human models in monocular image-driven 3D human animation. The authors propose a universal neural animation model that dispenses with LBS entirely, leveraging a Transformer-based motion regressor to directly map multimodal 2D control signals—such as images, keypoints, or sketches—into 3D Gaussian deformations. This formulation explicitly decouples global rigid motion from local non-rigid details. By integrating hybrid supervision with structural prior distillation, the model enables end-to-end training and jointly exploits both labeled data and unlabeled in-the-wild videos. To the best of our knowledge, this is the first method to achieve implicitly 2D-driven animatable 3D human generation, significantly enhancing motion realism while preserving visual fidelity, and demonstrating zero-shot generalization across identities and input modalities.

Abstract

Creating photorealistic, animatable 3D human avatars from monocular images still largely depends on Linear Blend Skinning (LBS) and parametric body models, which constrain expressivity and often introduce artifacts due to imperfect fitting. We propose LUNA, an LBS-free universal neural animation model that directly maps multiple 2D controls like images, keypoints, sketches, and unseen characters into 3D Gaussian deformations, bypassing explicit body fitting. At its core, a transformer-based motion regressor disentangles global rigid motion from fine-grained local dynamics to capture both coherent movement and subtle non-rigid effects. To resolve the inherent ambiguity of 2D-to-3D lifting while scaling beyond fitted datasets, we introduce hybrid supervision that distills soft structural priors from an LBS teacher and a loss that supports training on both limited fitted data and large in-the-wild unlabeled videos. Extensive experiments show LUNA achieves competitive visual fidelity compared to LBS-based approaches, while delivering realistic human motion and zero-shot cross-identity generalization across diverse driving modalities. To the best of our knowledge, LUNA is the first end-to-end 3D animatable model that supports implicit 2D driving.

Problem

• 3D human animation
• Linear Blend Skinning
• monocular images
• animatable avatars
• 2D-to-3D lifting

Innovation

• LBS-free animation
• 3D Gaussian deformations
• implicit 2D driving
• transformer-based motion regressor
• hybrid supervision