Designing Human and Generative AI Collaboration

2024-12-14

arXiv.org

AI Summary

Generative AI often compromises narrative quality, user satisfaction, and creative diversity in collaborative writing. Method: A controlled experiment compared two intervention timing paradigms—AI-first versus human-first—assessing narrative outputs via multidimensional automated metrics (engagement, quality, thematic diversity) and subjective user satisfaction scales. Contribution/Results: We provide the first empirical evidence that AI-only generation significantly degrades output quality and user satisfaction. In contrast, human-led ideation coupled with AI-assisted execution yields substantial improvements: +23% in narrative engagement, +31% in user satisfaction, and +40% in thematic diversity—effectively mitigating narrative homogenization. These findings establish “human-led, AI-co-creative” collaboration as a critical paradigm for preserving both creative quality and diversity. The study offers theoretical grounding and empirical validation for designing AI-augmented creative tools that prioritize human agency and expressive plurality.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: Generation

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

We examined the effectiveness of various human-AI collaboration designs on creative work. Through a human subjects experiment set in the context of creative writing, we found that while AI assistance improved productivity across all models, collaboration design significantly influenced output quality, user satisfaction, and content characteristics. Models incorporating human creative input delivered higher content interestingness and overall quality as well as greater task performer satisfaction compared to conditions where humans were limited to confirming AI's output. Increased AI involvement encouraged creators to explore beyond personal experience but also led to lower aggregate diversity in stories and genres among participants. However, this effect was mitigated through human participation in early creative tasks. These findings underscore the importance of preserving the human creative role to ensure quality, satisfaction, and creative diversity in human-AI collaboration.

Problem

• Human-AI Collaboration
• Creative Work
• Optimization

Innovation

• Human-AI Collaboration
• Creativity Diversity
• Work Quality Enhancement