Penny: Transition Network Analysis of Learner-Chatbot Interactions in Scaffolded EFL Writing

2026-07-16

AI Summary

This study unpacks the “black box” of human–AI interaction in generative AI–mediated English as a Foreign Language (EFL) writing tutoring by applying transition network analysis to model the temporal dynamics of over 4,500 dialogues (comprising more than 21,000 utterances) between Japanese learners and “Penny,” an LLM-powered chatbot. It identifies two dominant interaction patterns—“revision loops” and “dialogue loops”—and demonstrates that learners’ proficiency levels significantly shape their strategic engagement: higher-proficiency learners favor open-ended negotiation, whereas lower-proficiency learners rely on repetitive error correction. The findings empirically establish AI-assisted writing as a nonlinear, dialogic cognitive process, offering both methodological innovation and evidence-based insights for designing differentiated intelligent tutoring systems.

Abstract

Generative AI chatbots promise to transform English as a Foreign Language (EFL) writing by providing immediate, personalised feedback. However, their pedagogical value depends on how learners engage with them - a process often treated as a "black box." This study uses Transition Network Analysis to model the temporal dynamics of Japanese EFL learners using "Penny," an LLM-powered writing chatbot. Analysis of over 4,500 writing sessions and 21,000 chatbot interactions reveals two dominant behavioural loops: a "Revision Loop," where feedback leads directly to successful error correction, and a "Chat Loop," where learners engage in sustained dialogue with the chatbot following feedback. Crucially, EFL proficiency significantly shapes interaction: high-proficiency learners engage more in open dialogue and negotiation with the chatbot, while low-proficiency learners rely more heavily on repetitive corrective feedback cycles. The findings demonstrate that AI-scaffolded writing is a non-linear, dialogic process and highlight the need for differentiated chatbot design to move beyond simple error correction and foster deeper cognitive engagement for all learners.

Problem

• learner-chatbot interaction
• EFL writing
• scaffolding
• behavioral dynamics
• proficiency differences

Innovation

• Transition Network Analysis
• Generative AI chatbot
• EFL writing
• scaffolded interaction
• behavioral loops