Directive, Metacognitive or a Blend of Both? A Comparison of AI-Generated Feedback Types on Student Engagement, Confidence, and Outcomes

2025-10-22

AI Summary

This study investigates how AI-generated directive feedback, metacognitive feedback, and their hybrid combination affect student engagement, self-efficacy, and learning outcomes in programming and design education. Method: A semester-long randomized controlled trial was conducted using an adaptive educational platform that delivered personalized feedback and tracked students’ revision behaviors and final artifact quality. Contribution/Results: We propose and empirically validate a novel hybrid AI feedback framework that integrates immediate procedural guidance with reflective scaffolding. Results show that the hybrid feedback group exhibited significantly higher revision frequency than both the directive-only and metacognitive-only groups—indicating enhanced self-regulated learning. While all three groups reported high self-efficacy and produced artifacts of comparable final quality, only the hybrid condition uniquely fostered sustained iterative improvement. These findings provide empirical support for hybrid AI feedback as a promising paradigm in intelligent tutoring systems, advancing the design of pedagogically grounded, adaptive feedback mechanisms.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Intelligent Robots: Embodied AI
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Feedback is one of the most powerful influences on student learning, with extensive research examining how best to implement it in educational settings. Increasingly, feedback is being generated by artificial intelligence (AI), offering scalable and adaptive responses. Two widely studied approaches are directive feedback, which gives explicit explanations and reduces cognitive load to speed up learning, and metacognitive feedback which prompts learners to reflect, track their progress, and develop self-regulated learning (SRL) skills. While both approaches have clear theoretical advantages, their comparative effects on engagement, confidence, and quality of work remain underexplored. This study presents a semester-long randomised controlled trial with 329 students in an introductory design and programming course using an adaptive educational platform. Participants were assigned to receive directive, metacognitive, or hybrid AI-generated feedback that blended elements of both directive and metacognitive feedback. Results showed that revision behaviour differed across feedback conditions, with Hybrid prompting the most revisions compared to Directive and Metacognitive. Confidence ratings were uniformly high, and resource quality outcomes were comparable across conditions. These findings highlight the promise of AI in delivering feedback that balances clarity with reflection. Hybrid approaches, in particular, show potential to combine actionable guidance for immediate improvement with opportunities for self-reflection and metacognitive growth.

Problem

• Compares directive and metacognitive AI feedback effects
• Examines feedback impact on student engagement and confidence
• Evaluates hybrid feedback blending guidance with reflection prompts

Innovation

• Hybrid AI feedback combines directive and metacognitive approaches
• Adaptive educational platform delivers scalable personalized feedback
• Balanced feedback prompts revisions while maintaining student confidence