LEARN: A Story-Driven Layout-to-Image Generation Framework for STEM Instruction

2025-08-14

AI Summary

To address challenges in STEM education—including the difficulty of representing abstract concepts, attention fragmentation exacerbated by short-form media, and cognitively misaligned illustrations—this paper introduces the first layout-aware, narrative-driven diffusion model. The method formalizes page layout as a visual narrative scaffold, integrating semantic structure learning with cognitive scaffolding mechanisms. It employs layout-conditioned generation, contrastive vision–language alignment training, and prompt modulation to jointly optimize spatial organization, semantic coherence, and cognitive load in instructional illustrations. Evaluated on the BookCover dataset, the model significantly improves conceptual continuity and compatibility with curriculum knowledge graphs. This work establishes a unified generative modeling framework for AI-augmented educational content creation, advancing the integration of pedagogical principles into diffusion-based image synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

LEARN is a layout-aware diffusion framework designed to generate pedagogically aligned illustrations for STEM education. It leverages a curated BookCover dataset that provides narrative layouts and structured visual cues, enabling the model to depict abstract and sequential scientific concepts with strong semantic alignment. Through layout-conditioned generation, contrastive visual-semantic training, and prompt modulation, LEARN produces coherent visual sequences that support mid-to-high-level reasoning in line with Bloom's taxonomy while reducing extraneous cognitive load as emphasized by Cognitive Load Theory. By fostering spatially organized and story-driven narratives, the framework counters fragmented attention often induced by short-form media and promotes sustained conceptual focus. Beyond static diagrams, LEARN demonstrates potential for integration with multimodal systems and curriculum-linked knowledge graphs to create adaptive, exploratory educational content. As the first generative approach to unify layout-based storytelling, semantic structure learning, and cognitive scaffolding, LEARN represents a novel direction for generative AI in education. The code and dataset will be released to facilitate future research and practical deployment.

Problem

• Generates pedagogically aligned STEM illustrations using layout-aware diffusion
• Reduces cognitive load while supporting mid-to-high-level reasoning
• Counters fragmented attention with story-driven, spatially organized narratives

Innovation

• Layout-aware diffusion framework for STEM education
• Contrastive visual-semantic training for alignment
• Integration with multimodal educational systems