Handling Students Dropouts in an LLM-driven Interactive Online Course Using Language Models

2025-08-24

AI Summary

This study addresses student attrition in LLM-driven interactive online courses through three objectives: (1) identifying key dropout determinants; (2) building a high-accuracy predictive model; and (3) designing proactive intervention mechanisms. We propose CPADP—a Course-Progress-Aware Dropout Prediction framework—that integrates interaction log analysis, multi-agent collaboration, and LLM-based semantic understanding to model dynamic textual engagement patterns. Building upon CPADP, we develop a personalized email re-engagement agent to close the prediction–intervention loop. Evaluated on real-world courses with over 3,000 students, CPADP achieves 95.4% prediction accuracy, significantly outperforming baseline methods. Our contribution lies in being the first to jointly incorporate course-progress awareness with LLM-powered multi-agent intervention, establishing a novel paradigm for scalable, interpretable AI-driven educational interventions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Interactive online learning environments, represented by Massive AI-empowered Courses (MAIC), leverage LLM-driven multi-agent systems to transform passive MOOCs into dynamic, text-based platforms, enhancing interactivity through LLMs. This paper conducts an empirical study on a specific MAIC course to explore three research questions about dropouts in these interactive online courses: (1) What factors might lead to dropouts? (2) Can we predict dropouts? (3) Can we reduce dropouts? We analyze interaction logs to define dropouts and identify contributing factors. Our findings reveal strong links between dropout behaviors and textual interaction patterns. We then propose a course-progress-adaptive dropout prediction framework (CPADP) to predict dropouts with at most 95.4% accuracy. Based on this, we design a personalized email recall agent to re-engage at-risk students. Applied in the deployed MAIC system with over 3,000 students, the feasibility and effectiveness of our approach have been validated on students with diverse backgrounds.

Problem

• Identifying factors causing student dropouts in LLM-driven courses
• Predicting student dropouts using adaptive framework with high accuracy
• Reducing dropouts through personalized email recall interventions

Innovation

• LLM-driven multi-agent systems for interactive courses
• Course-progress-adaptive dropout prediction framework
• Personalized email recall agent for re-engagement