Tracing Prompt-Level Trajectories to Understand Student Learning with AI in Programming Education

2026-04-11

AI Summary

This study addresses the ambiguity surrounding AI tool usage in programming courses, which contributes to divergent learning outcomes and an unclear understanding of student interactions with large language models (LLMs). Analyzing dialogue logs and final code submissions from 163 students completing Python assignments, the research employs conversation analysis, code comparison, and trajectory clustering to systematically identify distinct student–AI interaction patterns—ranging from “full delegation” to “iterative refinement.” Findings reveal that while most students directly adopt AI-generated code, a subset engages in iterative prompting that facilitates scaffolded learning. These interaction trajectories are significantly associated with students’ self-regulated learning strategies, learning orientations, and course performance, offering empirical insights and pedagogical implications for human–AI collaborative learning in educational contexts.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

As AI tools such as ChatGPT enter programming classrooms, students encounter differing rules across courses and instructors, which shape how they use AI and leave them with unequal capabilities for leveraging it. We investigate how students engaged with AI in an introductory Python assignment, analyzing student-LLM chat histories and final code submissions from 163 students. We examined prompt-level strategies, traced trajectories of interaction, and compared AI-generated code with student submissions. We identified trajectories ranging from full delegation to iterative refinement, with hybrid forms in between. Although most students directly copied AI-generated code in their submission, many students scaffolded the code generation through iterative refinement. We also contrasted interaction patterns with assignment outcomes and course performance. Our findings show that prompting trajectories serve as promising windows into students' self-regulation and learning orientation. We draw design implications for educational AI systems that promote personalized and productive student-AI collaborative learning.

Problem

• AI in education
• student-AI interaction
• prompt-level trajectories
• programming education
• learning equity

Innovation

• prompt-level trajectories
• student-AI collaboration
• iterative refinement
• learning analytics
• educational AI