The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation

2026-09-13

AI Summary

研究通过分析用户与聊天机器人的自然对话，构建了故事生成提示的数据集，并提出了一个编辑类型框架来探索用户如何迭代编辑故事提示以探索叙事可能性。

Abstract

Large language models (LLMs) have changed the way people engage with stories. Drawing on public chatbot logs, we can see that when users generate stories, they iteratively edit their prompts to explore narrative possibilities, adjusting characters, redirecting plots, and swapping fictional universes. As aggregated data, these prompts represent rich traces of creative preference at scale. Yet story generation evaluation benchmarks rely on static, one-shot prompts that cannot capture this exploratory behavior. In this work, we study how users revise consecutive story prompts in the wild. Using a dataset of naturally occurring user-chatbot conversations, we construct WildStories, a sample of 275,635 story generation prompts (labeled with story format, prompt components, and explicitness), and WildEdits, a collection of 24,291 edit trees that model how users iteratively edit base story prompts and explore branching story possibilities. From these trees we develop a framework of edit types crossing four directions (adding, removing, changing, and extending) with fourteen targets (e.g., plot, character, genre). We then use our datasets and this framework to analyze user behavior in navigating narrative space via LLMs. Finally, we show how automated permutations based on the framework can be used for story generation benchmarking. Content Warning: This paper works with "wild" chatbot logs, which often include toxic and sexually explicit themes.

Problem

• story generation
• iterative editing
• narrative exploration
• large language models
• user behavior

Innovation

• Large language models
• Iterative editing
• Story generation
• Edit types framework
• Benchmarking