Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation

2026-06-26

AI Summary

Current vision-language models excel at static image-to-code tasks but struggle to capture temporal dynamics in video-to-code generation. To address this gap, this work introduces Animation2Code—the first temporally aware visual reasoning benchmark for video-to-code generation—comprising a large-scale dataset of web animation videos paired with executable HTML/CSS/JavaScript ground-truth code. The benchmark features two decoupled, human-aligned automatic evaluation metrics: appearance similarity and temporal similarity, enabling dual-axis assessment based on rendered outputs. Experimental results demonstrate that even state-of-the-art models, when fine-tuned or iteratively refined, fail to simultaneously achieve high fidelity in visual appearance and consistency in temporal behavior.

Abstract

While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present. To this end, we introduce Animation2Code, a benchmark for evaluating temporal visual reasoning via reconstructing executable web animation code from videos. Animation2Code consists of 1,069 web animation videos with diverse visual appearances and motion patterns, paired with corresponding HTML/CSS/JavaScript implementations. We propose two human-aligned metrics, appearance similarity and temporal similarity, which allow us to disentangle visual fidelity from temporal alignment when comparing rendered animations against ground-truth samples. Benchmarking state-of-the-art VLMs on this dataset shows that current VLMs struggle to maintain temporal consistency in reconstruction, even when achieving high appearance similarity, including under finetuning and iterative refinement settings. Code and data are available at https://anya-ji.github.io/animation2code-website .

Problem

• temporal visual reasoning
• video-to-code generation
• animation reconstruction
• vision-language models
• temporal consistency

Innovation

• temporal visual reasoning
• video-to-code generation
• animation reconstruction
• vision-language models
• human-aligned metrics