AI instructional agent improves student's perceived learner control and learning outcome: empirical evidence from a randomized controlled trial

2025-05-28

AI Summary

This study investigates how AI teaching agents influence students’ perceived sense of learning control and academic performance in moderately difficult expository courses. Method: A randomized controlled trial (RCT) was conducted, deploying a conversational AI teaching system integrating personalized pacing regulation, natural language question-answering, and real-time behavioral feedback, and comparing it against human instructors and MOOC chatbots. Contribution/Results: The AI agent significantly enhanced students’ perceived learning control (p < 0.01), accompanied by improved task completion efficiency and increased interaction frequency. Regression analysis confirmed that perceived control positively predicted post-test performance. This is the first RCT to empirically demonstrate that AI teaching agents improve learning outcomes by strengthening learners’ subjective agency—offering novel evidence for both intelligent educational agent design and the psychological mechanisms underpinning self-regulated learning.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Multiagent Systems: Agent/AI Theories and Architectures
• Humans and AI: Learning Human Values and Preferences

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

This study examines the impact of an AI instructional agent on students' perceived learner control and academic performance in a medium demanding course with lecturing as the main teaching strategy. Based on a randomized controlled trial, three instructional conditions were compared: a traditional human teacher, a self-paced MOOC with chatbot support, and an AI instructional agent capable of delivering lectures and responding to questions in real time. Students in the AI instructional agent group reported significantly higher levels of perceived learner control compared to the other groups. They also completed the learning task more efficiently and engaged in more frequent interactions with the instructional system. Regression analyzes showed that perceived learner control positively predicted post-test performance, with behavioral indicators such as reduced learning time and higher interaction frequency supporting this relationship. These findings suggest that AI instructional agents, when designed to support personalized pace and responsive interaction, can enhance both students' learning experience and learning outcomes.

Problem

• Impact of AI agent on learner control and performance
• Comparison of AI agent vs human teacher and MOOC
• AI agent enhances learning experience and outcomes

Innovation

• AI agent delivers lectures and answers questions
• Personalized pace and responsive interaction design
• Enhances learner control and academic performance