Scaffolding Metacognition in Programming Education: Understanding Student-AI Interactions and Design Implications

2025-11-06

AI Summary

This study addresses the gap in understanding how generative AI (e.g., ChatGPT) influences students’ metacognitive processes in programming education—moving beyond usability-focused evaluations to examine metacognitive support mechanisms. Drawing on over 10,000 student–AI dialogue logs collected over three years, complemented by student and instructor survey data, we systematically identify prompt and feedback patterns that either facilitate or hinder metacognitive engagement across the three core phases: planning, monitoring, and evaluation. We propose a pedagogically grounded AI teaching assistant design framework centered on “scaffolding—not supplanting—metacognition,” and derive actionable design principles. Results reveal widespread deficiencies in current AI tools’ metacognitive support. Our work provides both theoretical foundations and practical guidelines for developing intelligent programming tutors that foster autonomous learning and deep conceptual understanding.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Search and Optimization: Metareasoning and Metaheuristics
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Generative AI tools such as ChatGPT now provide novice programmers with unprecedented access to instant, personalized support. While this holds clear promise, their influence on students'metacognitive processes remains underexplored. Existing work has largely focused on correctness and usability, with limited attention to whether and how students'use of AI assistants supports or bypasses key metacognitive processes. This study addresses that gap by analyzing student-AI interactions through a metacognitive lens in university-level programming courses. We examined more than 10,000 dialogue logs collected over three years, complemented by surveys of students and educators. Our analysis focused on how prompts and responses aligned with metacognitive phases and strategies. Synthesizing these findings across data sources, we distill design considerations for AI-powered coding assistants that aim to support rather than supplant metacognitive engagement. Our findings provide guidance for developing educational AI tools that strengthen students'learning processes in programming education.

Problem

• Analyzing student-AI interactions through metacognitive lens in programming education
• Investigating whether AI assistants support or bypass students' metacognitive processes
• Designing AI coding tools to strengthen rather than replace metacognitive engagement

Innovation

• Analyzing student-AI interactions via metacognitive lens
• Using dialogue logs and surveys across multiple years
• Designing AI assistants to support metacognitive engagement