Do Students Write Better Post-AI Support? Effects of Generative AI Literacy and Chatbot Interaction Strategies on Multimodal Academic Writing

2025-07-06

AI Summary

This study investigates how generative AI literacy and chatbot interaction strategies—passive responsiveness versus active guidance—affect students’ multimodal academic writing competence and transferability. A controlled experimental design was employed, with writing performance assessed across five dimensions: insightfulness, image-text integration, structural organization, linguistic quality, and critical thinking. Ordered logistic regression and correlation analyses were conducted to examine predictive relationships. Results indicate that generative AI literacy significantly and positively predicts students’ post-AI independent writing proficiency. Notably, under passive responsiveness, high-literacy learners demonstrated superior transfer outcomes, underscoring the pivotal role of autonomous AI utilization. The study proposes an innovative “autonomy–scaffolding” dynamic balance pedagogical framework, offering empirically grounded, actionable guidelines for AI-enhanced multimodal writing instruction. (149 words)

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Generation
• Humans and AI: AI for Accessibility

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Academic writing increasingly involves multimodal tasks requiring students to integrate visual information and textual arguments. While generative AI (GenAI) tools, like ChatGPT, offer new pathways for supporting academic writing, little is known about how students' GenAI literacy influences their independent multimodal writing skills or how chatbot interaction strategies (passive reactive vs. proactive scaffolding) impact learning. This study examined 79 higher education students' multimodal academic writing performance using a comparative research design. Students completed writing tasks integrating visual data under two chatbot-assisted conditions (passive vs. proactive) and subsequently without AI assistance. Their writing performance was rigorously evaluated across five dimensions, including insightfulness, visual data integration, organisation, linguistic quality, and critical thinking. Ordinal logistic regression and correlation analyses revealed that higher levels of GenAI literacy significantly predicted stronger independent multimodal writing performance immediately after AI assistance removal, particularly for students using passive chatbots requiring active prompting. These results highlight the critical role of GenAI literacy and specific chatbot interaction strategies in shaping students' capacities for independent multimodal academic writing. Our findings emphasise the need for purposeful integration of GenAI literacy training into curricula and balancing external scaffolding support with autonomous learning opportunities. This research offers valuable recommendations for educators leveraging AI-enhanced pedagogies to optimise student writing outcomes and technological engagement strategies.

Problem

• How GenAI literacy affects students' independent multimodal writing skills
• Impact of chatbot interaction strategies on academic writing performance
• Balancing AI support with autonomous learning in writing curricula

Innovation

• Comparative study on passive vs proactive chatbot strategies
• GenAI literacy enhances independent multimodal writing skills
• Integration of GenAI training into academic curricula