How Problematic Writer-AI Interactions (Rather than Problematic AI) Hinder Writers' Idea Generation

2025-03-14

AI Summary

This study investigates how students’ interaction patterns with generative AI writing assistants influence creative idea generation, revealing that constraints stem not from AI capabilities per se, but from the quality of human–AI interaction. Guided by educational human–computer interaction theory, we conducted a qualitative case study integrating behavioral log analysis and cognitive retrospective interviews. Our findings—novel in this domain—demonstrate that the writer–AI interaction pattern (e.g., exploratory questioning, iterative restructuring vs. passive copying, superficial editing), rather than AI system type, is the primary determinant of constructive learning outcomes. High-agency interactions significantly increase idea output, whereas low-agency interactions impede cognitive gains. We thus propose a paradigm shift—from optimizing AI response quality to *eliciting deep, agentic interaction*—offering both theoretical grounding and actionable design principles for AI-enhanced creative writing education.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Humans and AI: Interaction Techniques and Devices
• Intelligent Robots: Human-Robot Interaction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Writing about a subject enriches writers' understanding of that subject. This cognitive benefit of writing -- known as constructive learning -- is essential to how students learn in various disciplines. However, does this benefit persist when students write with generative AI writing assistants? Prior research suggests the answer varies based on the type of AI, e.g., auto-complete systems tend to hinder ideation, while assistants that pose Socratic questions facilitate it. This paper adds an additional perspective. Through a case study, we demonstrate that the impact of genAI on students' idea development depends not only on the AI but also on the students and, crucially, their interactions in between. Students who proactively explored ideas gained new ideas from writing, regardless of whether they used auto-complete or Socratic AI assistants. Those who engaged in prolonged, mindless copyediting developed few ideas even with a Socratic AI. These findings suggest opportunities in designing AI writing assistants, not merely by creating more thought-provoking AI, but also by fostering more thought-provoking writer-AI interactions.

Problem

• Impact of AI on students' idea generation
• Role of writer-AI interactions in learning
• Designing AI to enhance constructive learning

Innovation

• Explores impact of writer-AI interactions on ideation
• Highlights proactive idea exploration over AI type
• Suggests designing AI to enhance writer engagement