ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframing

2025-08-14

AI Summary

In traditional cartoon animation production, keyframe drawing, in-between generation, and coloring are typically decoupled, leading to error accumulation and visual artifacts. This paper proposes an end-to-end paradigm that jointly performs in-betweening and coloring as a unified “generative post-keyframing” stage, synthesizing high-quality cartoon videos from only extremely sparse hand-drawn sketches and minimal color references. Our core contributions include: (1) a sparse sketch injection mechanism enabling cross-frame spatiotemporal alignment; (2) a cartoon-domain-specific spatial low-rank adapter enhancing stylistic consistency and detail fidelity; and (3) a lightweight diffusion architecture built upon video foundation models. Evaluated on our newly established real-world benchmark PKBench, our method significantly outperforms state-of-the-art approaches, achieving substantial improvements in visual quality, motion coherence, and production efficiency—demonstrating a highly practical AI-assisted animation generation pathway.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Traditional cartoon and anime production involves keyframing, inbetweening, and colorization stages, which require intensive manual effort. Despite recent advances in AI, existing methods often handle these stages separately, leading to error accumulation and artifacts. For instance, inbetweening approaches struggle with large motions, while colorization methods require dense per-frame sketches. To address this, we introduce ToonComposer, a generative model that unifies inbetweening and colorization into a single post-keyframing stage. ToonComposer employs a sparse sketch injection mechanism to provide precise control using keyframe sketches. Additionally, it uses a cartoon adaptation method with the spatial low-rank adapter to tailor a modern video foundation model to the cartoon domain while keeping its temporal prior intact. Requiring as few as a single sketch and a colored reference frame, ToonComposer excels with sparse inputs, while also supporting multiple sketches at any temporal location for more precise motion control. This dual capability reduces manual workload and improves flexibility, empowering artists in real-world scenarios. To evaluate our model, we further created PKBench, a benchmark featuring human-drawn sketches that simulate real-world use cases. Our evaluation demonstrates that ToonComposer outperforms existing methods in visual quality, motion consistency, and production efficiency, offering a superior and more flexible solution for AI-assisted cartoon production.

Problem

• Unifies inbetweening and colorization to reduce manual effort
• Handles sparse inputs for precise motion and color control
• Adapts video foundation models for cartoon production efficiency

Innovation

• Unifies inbetweening and colorization into one stage
• Uses sparse sketch injection for precise control
• Adapts video model to cartoons with spatial low-rank adapter