Automatic Large Language Models Creation of Interactive Learning Lessons

2025-06-20

AI Summary

Automating the generation of high-quality training curricula for novice tutors in online middle-school mathematics instruction remains challenging due to the complexity of pedagogical alignment and contextual fidelity. Method: This study proposes a task-decomposition prompting strategy integrating retrieval-augmented generation (RAG), multi-stage prompt engineering, and evidence-based instructional design principles to construct interactive, context-aware tutor training modules—specifically targeting student autonomy development, help-seeking behavior scaffolding, and classroom participation motivation. Contribution/Results: Compared to single-step generation, the approach significantly improves curriculum structural coherence and pedagogical appropriateness; expert double-blind evaluation shows average gains of 23% in clarity and practicality scores. It represents the first empirical validation of human-AI collaborative generation of rigorous, scalable teacher professional development content, demonstrating strong potential for real-world educational deployment.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We explore the automatic generation of interactive, scenario-based lessons designed to train novice human tutors who teach middle school mathematics online. Employing prompt engineering through a Retrieval-Augmented Generation approach with GPT-4o, we developed a system capable of creating structured tutor training lessons. Our study generated lessons in English for three key topics: Encouraging Students' Independence, Encouraging Help-Seeking Behavior, and Turning on Cameras, using a task decomposition prompting strategy that breaks lesson generation into sub-tasks. The generated lessons were evaluated by two human evaluators, who provided both quantitative and qualitative evaluations using a comprehensive rubric informed by lesson design research. Results demonstrate that the task decomposition strategy led to higher-rated lessons compared to single-step generation. Human evaluators identified several strengths in the LLM-generated lessons, including well-structured content and time-saving potential, while also noting limitations such as generic feedback and a lack of clarity in some instructional sections. These findings underscore the potential of hybrid human-AI approaches for generating effective lessons in tutor training.

Problem

• Automate creation of interactive tutor training lessons
• Generate scenario-based lessons for middle school math tutors
• Evaluate effectiveness of task decomposition in lesson generation

Innovation

• Retrieval-Augmented Generation with GPT-4o
• Task decomposition prompting strategy
• Hybrid human-AI lesson evaluation