Plan More, Debug Less: Applying Metacognitive Theory to AI-Assisted Programming Education

2025-09-03

AI Summary

Current AI-powered programming education tools lack explicit metacognitive scaffolding, limiting students’ self-regulated learning. Method: This study proposes the first metacognitively grounded, three-stage prompting framework—aligned with Flavell’s planning–monitoring–evaluation model—to dynamically support students’ code planning, debugging, and optimization via generative AI. An empirical study was conducted with 102 data science students, integrating educational data mining and behavioral log analysis. Contribution/Results: Planning prompts significantly improved academic performance and achieved the highest acceptance rate; debugging support was most frequently adopted during high-complexity tasks; optimization prompts saw lowest uptake. This work establishes the first structured integration of metacognitive theory into generative AI–enhanced programming instruction, yielding a theoretically grounded, empirically validated design paradigm for intelligent educational systems.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

The growing adoption of generative AI in education highlights the need to integrate established pedagogical principles into AI-assisted learning environments. This study investigates the potential of metacognitive theory to inform AI-assisted programming education through a hint system designed around the metacognitive phases of planning, monitoring, and evaluation. Upon request, the system can provide three types of AI-generated hints--planning, debugging, and optimization--to guide students at different stages of problem-solving. Through a study with 102 students in an introductory data science programming course, we find that students perceive and engage with planning hints most highly, whereas optimization hints are rarely requested. We observe a consistent association between requesting planning hints and achieving higher grades across question difficulty and student competency. However, when facing harder tasks, students seek additional debugging but not more planning support. These insights contribute to the growing field of AI-assisted programming education by providing empirical evidence on the importance of pedagogical principles in AI-assisted learning.

Problem

• Applying metacognitive theory to AI-assisted programming education
• Investigating AI-generated hints for planning, debugging, and optimization
• Evaluating student engagement with different hint types

Innovation

• Metacognitive theory-based hint system design
• AI-generated planning, debugging, optimization hints
• Planning hints association with higher grades