UNICA: A Unified Neural Framework for Controllable 3D Avatars

2026-04-03

AI Summary

This work proposes UNICA, the first end-to-end neural generative framework for controllable 3D digital humans that eliminates the need for explicit skeletal rigs or physics-based simulation. UNICA unifies motion planning, skinning, dynamics modeling, and rendering into a single pipeline. Given keyboard-style game inputs, a motion-conditioned diffusion model generates 2D positional maps, which are then mapped into a 3D Gaussian splatting space via a point transformer to enable free-viewpoint, high-fidelity, real-time rendering. The framework supports ultra-long autoregressive generation and naturally captures intricate dynamics of hair and loose clothing, significantly simplifying the traditional multi-stage pipeline while enhancing both interactivity and realism.

Abstract

Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce UNICA (UNIfied neural Controllable Avatar), a skeleton-free generative model that unifies all avatar control components into a single neural framework. Given keyboard inputs akin to video game controls, UNICA generates the next frame of a 3D avatar's geometry through an action-conditioned diffusion model operating on 2D position maps. A point transformer then maps the resulting geometry to 3D Gaussian Splatting for high-fidelity free-view rendering. Our approach naturally captures hair and loose clothing dynamics without manually designed physical simulation, and supports extra-long autoregressive generation. To the best of our knowledge, UNICA is the first model to unify the workflow of "motion planning, rigging, physical simulation, and rendering". Code is released at https://github.com/zjh21/UNICA.

Problem

• controllable 3D avatars
• motion planning
• rigging
• physical simulation
• unified framework

Innovation

• unified neural avatar
• skeleton-free generation
• action-conditioned diffusion
• 3D Gaussian Splatting
• autoregressive 3D animation