Can AI support student engagement in classroom activities in higher education?

2025-06-22

AI Summary

Low student engagement and insufficient instructor-student interaction persist in large-scale undergraduate computer science courses. Method: This study conducts the first controlled experiment in a real-world, large-enrollment teaching setting to investigate the impact of large language model–driven conversational AI (CAI)—exemplified by ChatGPT—on learning engagement. CAI is deeply integrated into classroom activity design and interaction protocols, enabling real-time feedback and content-based discussion. Results: CAI significantly enhances in-class participation, particularly increasing students’ willingness to interact and their cognitive and behavioral investment in learning within large cohorts. This work provides empirical evidence for AI-enhanced higher education and establishes a reproducible, rigorously evaluable controlled-experiment paradigm for deploying CAI in authentic instructional environments. It thus offers both methodological guidance and actionable implementation pathways for the intelligent transformation of education.

Technology Category

• Humans and AI: Human-Computer Interaction
• Machine Learning: Causal Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Lucrative career prospects and creative opportunities often attract students to enroll in computer science majors and pursue advanced studies in the field. Consequently, there has been a significant surge in enrollment in computer science courses, resulting in large class sizes that can range from hundreds to even thousands of students. A common challenge in such large classrooms is the lack of engagement between students and both the instructor and the learning material. However, with advancements in technology and improvements in large language models (LLMs), there is a considerable opportunity to utilize LLM-based AI models, such as conversational artificial intelligence (CAI), to enhance student engagement with learning content in large classes. To explore the potential of CAI to support engagement, especially with learning content, we designed an activity in a software Engineering course (with a large class size) where students used CAI for an in-class activity. We conducted a within-subject investigation in a large classroom at a US university where we compared student engagement during an in-class activity that used CAI tool vs. one without CAI tool. The CAI tool we used was ChatGPT due to its widespread popularity and familiarity. Our results indicate that CAI (ChatGPT) has the potential to support engagement with learning content during in-class activities, especially in large class sizes. We further discuss the implications of our findings.

Problem

• AI enhances student engagement in large classes
• Addressing lack of interaction in computer science courses
• Evaluating ChatGPT's impact on learning content engagement

Innovation

• Used ChatGPT for student engagement
• Conducted within-subject CAI investigation
• Applied LLM-based AI in large classes