SketchKeyAnime: Reference-anchored Sparse Key-Sketch Animation Synthesis

2026-06-18

AI Summary

Traditional animation generation relies heavily on dense frame-wise conditioning or complete sketch sequences, making it challenging to simultaneously ensure structural controllability, appearance consistency, and temporal coherence under sparse inputs. This work proposes a video diffusion-based animation synthesis framework that requires only a single reference RGB image and a few timestamped key sketches. It employs a dual-branch conditional encoder to fuse local geometric constraints with semantic temporal context. The method introduces a novel sketch cross-attention mechanism and a learnable gating module to effectively align reference image and sketch information, along with an adaptive weighted loss that enhances supervision on keyframes and line-art regions. Evaluated on the Sakuga-42M anime subset, the approach outperforms the best baseline by reducing EDMD by 31.9% and FVD by 9.5%, achieving state-of-the-art performance in sketch fidelity, temporal coherence, and multiple quantitative metrics.

Abstract

Traditional animation production relies heavily on manual drawing and iterative refinement, particularly for key-pose design, in-betweening, and character coloring. While existing animation and video generation methods have made notable progress, they typically depend on RGB boundary frames, dense frame-wise conditions, or complete sketch sequences, limiting their applicability under low-cost input conditions. We present SketchKeyAnime, a video diffusion framework for generating structurally controllable, appearance-consistent, and temporally coherent animations from sparse key-sketch inputs. Given a single reference RGB image and a few temporally indexed key sketches, SketchKeyAnime introduces a dual-branch conditioning mechanism to encode local geometric constraints alongside semantic-temporal context. It leverages Sketch Cross Attention to fuse reference image and sketch conditions with learnable gating, and incorporates an Adaptive Weighted Loss to strengthen supervision on key-sketch frames and line-art regions. Experimental results on the Aesthetic subset of Sakuga-42M show that our approach consistently outperforms representative animation interpolation and sketch-guided generation baselines. Compared to the best-performing baseline, SketchKeyAnime reduces EDMD by 31.9\% and FVD by 9.5\%, demonstrating superior sketch fidelity and temporal coherence, while achieving the best overall performance across most quantitative metrics. These results validate the proposed framework and highlight its potential for low-cost, highly controllable animation creation.

Problem

• animation synthesis
• sparse key-sketch
• low-cost input
• temporal coherence
• structural control

Innovation

• sparse key-sketch animation
• video diffusion
• Sketch Cross Attention
• adaptive weighted loss
• reference-anchored generation