PedaCo-Gen: Scaffolding Pedagogical Agency in Human-AI Collaborative Video Authoring

2026-02-23

AI Summary

This study addresses the limitation of current text-to-video generation models, which prioritize visual fidelity over pedagogical effectiveness and thus struggle to support educators in creating instructionally sound videos aligned with cognitive theories. Drawing on Mayer’s Cognitive Theory of Multimedia Learning (CTML), the authors propose a human-AI collaborative video generation framework that introduces an instructional-principle-guided intermediate representation (IR) mechanism. In this approach, AI serves as a metacognitive scaffold—augmenting rather than replacing teachers’ professional judgment. The system integrates CTML-driven prompt engineering, IR modeling, and interactive feedback loops to substantially enhance the instructional quality of generated videos across diverse topics and CTML principles. Evaluations by 23 educational experts demonstrate strong performance in both production efficiency (M = 4.26) and instructional effectiveness (M = 4.04).

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

While advancements in Text-to-Video (T2V) generative AI offer a promising path toward democratizing content creation, current models are often optimized for visual fidelity rather than instructional efficacy. This study introduces PedaCo-Gen, a pedagogically-informed human-AI collaborative video generating system for authoring instructional videos based on Mayer's Cognitive Theory of Multimedia Learning (CTML). Moving away from traditional "one-shot" generation, PedaCo-Gen introduces an Intermediate Representation (IR) phase, enabling educators to interactively review and refine video blueprints-comprising scripts and visual descriptions-with an AI reviewer. Our study with 23 education experts demonstrates that PedaCo-Gen significantly enhances video quality across various topics and CTML principles compared to baselines. Participants perceived the AI-driven guidance not merely as a set of instructions but as a metacognitive scaffold that augmented their instructional design expertise, reporting high production efficiency (M=4.26) and guide validity (M=4.04). These findings highlight the importance of reclaiming pedagogical agency through principled co-creation, providing a foundation for future AI authoring tools that harmonize generative power with human professional expertise.

Problem

• Text-to-Video
• instructional efficacy
• pedagogical agency
• human-AI collaboration
• multimedia learning

Innovation

• Pedagogical Agency
• Human-AI Collaboration
• Intermediate Representation
• Cognitive Theory of Multimedia Learning
• Instructional Video Generation