Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners

2025-03-01

AI Summary

This study investigates the mechanisms through which AI-generated text (e.g., ChatGPT) influences English writing quality among EFL learners, with particular attention to how academic proficiency moderates human–AI co-writing efficacy. Method: Drawing on empirical data from 59 Hong Kong secondary students, the research employs content analysis, multiple linear regression, and cluster analysis. Contribution/Results: Total word count—comprising both human-authored and AI-generated text—is identified as the strongest predictor of writing quality; however, its impact is significantly moderated by students’ autonomous writing competence and their patterns of AI interaction. Crucially, high-achieving students integrate AI output more effectively, whereas low-achieving students, lacking AI literacy and targeted writing instruction, tend toward superficial reliance—exacerbating existing proficiency gaps. The study issues an equity-oriented caution: “AI empowerment must be grounded in foundational writing competence and mediated by pedagogical intervention.” It provides empirically grounded guidance for responsible integration of generative AI in EFL writing instruction.

Technology Category

• Humans and AI: Emotional Intelligence
• Natural Language Processing: Generation
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Generative artificial intelligence (AI) chatbots, such as ChatGPT, are reshaping how English as a foreign language (EFL) students write since students can compose texts by integrating their own words with AI-generated text. This study investigated how 59 Hong Kong secondary school students with varying levels of academic achievement interacted with AI-generated text to compose a feature article, exploring whether any interaction patterns benefited the overall quality of the article. Through content analysis, multiple linear regression and cluster analysis, we found the overall number of words -- whether AI- or human-generated -- is the main predictor of writing quality. However, the impact varies by students' competence to write independently, for instance, by using their own words accurately and coherently to compose a text, and to follow specific interaction patterns with AI-generated text. Therefore, although composing texts with human words and AI-generated text may become prevalent in EFL writing classrooms, without educators' careful attention to EFL writing pedagogy and AI literacy, high-achieving students stand to benefit more from using AI-generated text than low-achieving students.

Problem

• Impact of AI-generated text on EFL writing quality
• Interaction patterns between students and AI-generated text
• Varied benefits of AI text based on student competence

Innovation

• AI-generated text integration in EFL writing
• Content and cluster analysis for interaction patterns
• Multiple linear regression predicts writing quality