Code2Video: A Code-centric Paradigm for Educational Video Generation

2025-10-01

AI Summary

Existing generative models struggle to meet the stringent requirements of professional educational videos—namely, domain-specific accuracy, visually structured presentation, and logical coherence. To address this, we propose Code2Video, the first educational video generation framework centered on executable Python code, which explicitly encodes content structure, visual layout, and transition logic. Our method introduces a novel tri-agent collaboration mechanism (planning–coding–review), a vision-language model (VLM) evaluator enhanced with visual anchors, and TeachQuiz—a quantitative metric designed specifically for pedagogical effectiveness. Integrating instruction-driven code generation, scope-aware automatic code repair, and end-to-end code-to-video rendering, Code2Video significantly improves controllability and interpretability. Evaluated on the MMMC benchmark, it outperforms baseline methods by 40% and produces video quality approaching that of human-crafted tutorials.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

While recent generative models advance pixel-space video synthesis, they remain limited in producing professional educational videos, which demand disciplinary knowledge, precise visual structures, and coherent transitions, limiting their applicability in educational scenarios. Intuitively, such requirements are better addressed through the manipulation of a renderable environment, which can be explicitly controlled via logical commands (e.g., code). In this work, we propose Code2Video, a code-centric agent framework for generating educational videos via executable Python code. The framework comprises three collaborative agents: (i) Planner, which structures lecture content into temporally coherent flows and prepares corresponding visual assets; (ii) Coder, which converts structured instructions into executable Python codes while incorporating scope-guided auto-fix to enhance efficiency; and (iii) Critic, which leverages vision-language models (VLM) with visual anchor prompts to refine spatial layout and ensure clarity. To support systematic evaluation, we build MMMC, a benchmark of professionally produced, discipline-specific educational videos. We evaluate MMMC across diverse dimensions, including VLM-as-a-Judge aesthetic scores, code efficiency, and particularly, TeachQuiz, a novel end-to-end metric that quantifies how well a VLM, after unlearning, can recover knowledge by watching the generated videos. Our results demonstrate the potential of Code2Video as a scalable, interpretable, and controllable approach, achieving 40% improvement over direct code generation and producing videos comparable to human-crafted tutorials. The code and datasets are available at https://github.com/showlab/Code2Video.

Problem

• Generating professional educational videos with precise visual structures
• Addressing coherent transitions and disciplinary knowledge requirements
• Producing scalable and controllable educational content via code

Innovation

• Code-centric agent framework for educational video generation
• Three collaborative agents structure content and code
• Vision-language models refine layout and ensure clarity