Evaluating Quality of Gaming Narratives Co-created with AI

2025-09-04

AI Summary

This study addresses the lack of systematic quality assessment methods for AI-assisted game narrative generation. To bridge this gap, we propose the first multidimensional evaluation framework integrating expert consensus and player experience. Employing a three-round Delphi process with 21 narrative design experts, we synthesized established story quality dimensions from the literature and—novel in this domain—applied the Kano model to quantitatively classify narrative elements by their impact on player satisfaction (e.g., must-be, one-dimensional, attractive). The resulting framework encompasses core dimensions including narrative coherence, character believability, and player agency, accompanied by an actionable practitioner guide prioritizing high-impact narrative features. Our approach significantly improves design efficiency and player experience in AI-human co-creative storytelling, establishing a methodological foundation for trustworthy generative AI integration in game narrative development.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Game Theory and Economic Paradigms: Cooperative Game Theory
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

This paper proposes a structured methodology to evaluate AI-generated game narratives, leveraging the Delphi study structure with a panel of narrative design experts. Our approach synthesizes story quality dimensions from literature and expert insights, mapping them into the Kano model framework to understand their impact on player satisfaction. The results can inform game developers on prioritizing quality aspects when co-creating game narratives with generative AI.

Problem

• Evaluating quality of AI-generated game narratives
• Mapping story quality dimensions to player satisfaction
• Prioritizing quality aspects for AI co-created narratives

Innovation

• Delphi expert panel for narrative evaluation
• Kano model mapping story quality dimensions
• Framework prioritizes AI co-created narrative satisfaction