How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay Drafts

2026-07-16

AI Summary

This study addresses the misalignment between the pedagogical appropriateness of AI-generated written corrective feedback (WCF) and learners’ actual needs, despite its scalability. Deploying a large language model (LLM)-based WCF system in an English writing course involving nearly 2,000 undergraduate students, the research collected over 20,000 draft essays and developed a multidimensional, learner-centered evaluation framework grounded in external criteria. Dual validation was achieved through both instructor ratings and student feedback. Findings reveal a significant discrepancy between expert assessments and student perceptions, underscoring the limitations of relying solely on traditional expert evaluations. The results emphasize that the design of AI-powered educational tools must integrate the learner perspective to enhance pedagogical effectiveness.

Abstract

This study examines feedback in English as a Foreign Language (EFL) writing contexts, focusing on written corrective feedback (WCF). Large language models (LLMs) can provide WCF at scale, but aligning them with pedagogical best practices remains an ongoing challenge. WCF meeting criteria like factuality or relevance may still be unsuitable for learning contexts, highlighting the need for extrinsic evaluation based on the learner's perspective. We deployed WCF systems in a university-level EFL class with nearly 2,000 students, collecting over 20,000 drafts. We evaluated the generated WCF from two perspectives: intrinsic evaluation by experienced English teachers using a rubric, and extrinsic evaluation via student feedback and engagement metrics. Results revealed low alignment between teacher expert ratings and student feedback. These findings suggest that traditional expert evaluation alone may not fully capture WCF's usability or helpfulness from the learner's perspective, highlighting the importance of learner-centered evaluation frameworks for AI-based applications in language education.

Problem

• written corrective feedback
• AI-generated feedback
• EFL writing
• learner-centered evaluation
• extrinsic evaluation

Innovation

• written corrective feedback
• large language models
• learner-centered evaluation
• extrinsic evaluation
• EFL writing