The Reel Deal: Designing and Evaluating LLM-Generated Short-Form Educational Videos

2025-09-07

AI Summary

Short educational videos are increasingly prevalent, yet their manual production is costly, and the pedagogical efficacy, usability, and trustworthiness of AI-generated alternatives remain poorly understood. This study introduces ReelsEd—a novel LLM-driven system for automatically generating pedagogically aligned microlearning videos from instructor-created long-form content. ReelsEd leverages large language models to distill key concepts, generate structured scripts, and—via a web-based interface—automatically segment, resequence, and annotate videos with interactive navigation, thereby supporting learner autonomy and instructional objectives. In a controlled experiment with 62 undergraduate students, ReelsEd significantly improved learning engagement (+32%), post-test performance (+24%), and task efficiency (−19% completion time), without increasing cognitive load. Participants also reported high levels of trust across clarity, usability, and practical utility dimensions—marking the first systematic evaluation of LLM-based educational video generation across learning outcomes, usability, and trust.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Short-form videos are gaining popularity in education due to their concise and accessible format that enables microlearning. Yet, most of these videos are manually created. Even for those automatically generated using artificial intelligence (AI), it is not well understood whether or how they affect learning outcomes, user experience, and trust. To address this gap, we developed ReelsEd, which is a web-based system that uses large language models (LLMs) to automatically generate structured short-form video (i.e., reels) from lecture long-form videos while preserving instructor-authored material. In a between-subject user study with 62 university students, we evaluated ReelsEd and demonstrated that it outperformed traditional long-form videos in engagement, quiz performance, and task efficiency without increasing cognitive load. Learners expressed high trust in our system and valued its clarity, usefulness, and ease of navigation. Our findings point to new design opportunities for integrating generative AI into educational tools that prioritize usability, learner agency, and pedagogical alignment.

Problem

• Evaluating AI-generated short videos' impact on learning outcomes
• Assessing user experience and trust in automated educational content
• Designing LLM-based systems for microlearning from long lectures

Innovation

• LLM-generated structured short-form videos
• Web-based system preserving instructor-authored content
• Enhancing engagement and performance without cognitive load