Kling-MotionControl Technical Report

2026-03-03

AI Summary

This work proposes a unified DiT-based framework for high-fidelity, controllable, and cross-identity generalizable full-body character animation. To address the heterogeneity among body, facial, and hand motions, it adopts a divide-and-conquer modeling strategy, integrating identity-agnostic learning with a refined identity injection mechanism to achieve both structural stability and rich motion detail. The approach innovatively combines semantic-level motion understanding, text-responsive control, and multi-region heterogeneous modeling, enabling precise animation generation for open-domain characters—including both realistic and cartoon styles—and introduces a subject library mechanism to enhance reference context utilization. Through multi-stage distillation, inference speed is accelerated by over 10×, and human preference evaluations demonstrate superior performance against leading commercial and open-source alternatives in overall motion control, generalization, and visual quality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Character animation aims to generate lifelike videos by transferring motion dynamics from a driving video to a reference image. Recent strides in generative models have paved the way for high-fidelity character animation. In this work, we present Kling-MotionControl, a unified DiT-based framework engineered specifically for robust, precise, and expressive holistic character animation. Leveraging a divide-and-conquer strategy within a cohesive system, the model orchestrates heterogeneous motion representations tailored to the distinct characteristics of body, face, and hands, effectively reconciling large-scale structural stability with fine-grained articulatory expressiveness. To ensure robust cross-identity generalization, we incorporate adaptive identity-agnostic learning, facilitating natural motion retargeting for diverse characters ranging from realistic humans to stylized cartoons. Simultaneously, we guarantee faithful appearance preservation through meticulous identity injection and fusion designs, further supported by a subject library mechanism that leverages comprehensive reference contexts. To ensure practical utility, we implement an advanced acceleration framework utilizing multi-stage distillation, boosting inference speed by over 10x. Kling-MotionControl distinguishes itself through intelligent semantic motion understanding and precise text responsiveness, allowing for flexible control beyond visual inputs. Human preference evaluations demonstrate that Kling-MotionControl delivers superior performance compared to leading commercial and open-source solutions, achieving exceptional fidelity in holistic motion control, open domain generalization, and visual quality and coherence. These results establish Kling-MotionControl as a robust solution for high-quality, controllable, and lifelike character animation.

Problem

• character animation
• motion retargeting
• cross-identity generalization
• appearance preservation
• holistic motion control

Innovation

• DiT-based animation
• heterogeneous motion representation
• identity-agnostic learning
• multi-stage distillation
• text-driven motion control