From Formulas to Figures: How Visual Elements Impact User Interactions in Educational Videos

2025-05-03

AI Summary

This study investigates how visual complexity—specifically fine-grained visual objects such as mathematical formulas, chemical structures, and diagrams—in educational videos affects learner engagement behaviors (pausing, replaying, session abandonment). Leveraging 25 authentic physics and chemistry instructional videos, we construct the first fine-grained visual object taxonomy for educational video analysis, integrating object detection, human-verified annotation, and multi-dimensional behavioral log analytics. Our key empirical finding is that textual visual element complexity exhibits significant positive correlations with pause frequency, replay rate, and dropout rate; a one-level increase in visual complexity raises session abandonment risk by approximately 37%. These results uncover critical links between visual design principles and learning interaction dynamics, offering data-driven foundations for adaptive educational video optimization. The source code and annotation framework are publicly released.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Web data visualization
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Educational videos have become increasingly relevant in today's learning environments. While prior research in laboratory studies has provided valuable insights, analyzing real-world interaction data can enhance our understanding of authentic user behavior. Previous studies have investigated technical aspects, such as the influence of cuts on pausing behavior, but the impact of visual complexity remains understudied. In this paper, we address this gap and propose a novel approach centered on visual complexity, defined as the number of visually distinguishable and meaningful elements in a video frame, such as mathematical equations, chemical formulas, or graphical representations. Our study introduces a fine-grained taxonomy of visual objects in educational videos, expanding on previous classifications. Applying this taxonomy to 25 videos from physics and chemistry, we examine the relationship between visual complexity and user behavior, including pauses, in-video navigation, and session dropouts. The results indicate that increased visual complexity, especially of textual elements, correlates with more frequent pauses, rewinds, and dropouts. The results offer a deeper understanding of how video design affects user behavior in real-world scenarios. Our work has implications for optimizing educational videos, particularly in STEM fields. We make our code publicly available (https://github.com/TIBHannover/from_formulas_to_figures).

Problem

• Investigates impact of visual complexity on user behavior in educational videos
• Analyzes relationship between visual elements and pauses, rewinds, dropouts
• Proposes taxonomy for visual objects to optimize STEM video design

Innovation

• Defines visual complexity via distinguishable elements
• Introduces taxonomy for visual objects in videos
• Links visual complexity to user interaction behaviors