TutorUp: What If Your Students Were Simulated? Training Tutors to Address Engagement Challenges in Online Learning

2025-02-22

AI Summary

Novice teachers face persistent challenges in sustaining student engagement during online instruction. Method: This study introduces a large language model (LLM)-based simulation training system that uniquely integrates empirically grounded learning science strategies with high-fidelity LLM-driven student avatars, enabling dual feedback—real-time interaction and asynchronous reflection. The system employs contextualized prompt engineering, educational dialogue modeling, and a structured pedagogical strategy knowledge base to generate dynamic, diverse remote teaching scenarios. Contribution/Results: A user study (N=16) demonstrates statistically significant improvements over baseline systems: enhanced effectiveness in applying instructional strategies (p<0.01) and markedly higher system usability, with a 32% increase in System Usability Scale (SUS) scores. The system effectively strengthens novice teachers’ capacity to identify, respond to, and regulate online student engagement challenges.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

With the rise of online learning, many novice tutors lack experience engaging students remotely. We introduce TutorUp, a Large Language Model (LLM)-based system that enables novice tutors to practice engagement strategies with simulated students through scenario-based training. Based on a formative study involving two surveys (N1=86, N2=102) on student engagement challenges, we summarize scenarios that mimic real teaching situations. To enhance immersion and realism, we employ a prompting strategy that simulates dynamic online learning dialogues. TutorUp provides immediate and asynchronous feedback by referencing tutor-students online session dialogues and evidence-based teaching strategies from learning science literature. In a within-subject evaluation (N=16), participants rated TutorUp significantly higher than a baseline system without simulation capabilities regarding effectiveness and usability. Our findings suggest that TutorUp provides novice tutors with more effective training to learn and apply teaching strategies to address online student engagement challenges.

Problem

• Training novice tutors online
• Simulating student engagement scenarios
• Enhancing teaching strategies effectiveness

Innovation

• LLM-based simulated student training
• Dynamic online learning dialogues simulation
• Immediate feedback with evidence-based strategies