Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Generation

2026-01-23

AI Summary

This work addresses the lack of narrative theory–grounded training and evaluation mechanisms in existing automatic story generation approaches, which often rely heavily on limited human annotations. It proposes a linguistically principled, subjectivity-aware post-training paradigm by integrating Todorov’s narrative equilibrium theory into a reinforcement learning framework for the first time. Specifically, large language models (7B/14B) serve as critics to generate human-aligned reward signals via the d-RLAIF algorithm, with evaluations conducted using Gemini-3-Flash. Experimental results demonstrate that the proposed method significantly outperforms supervised fine-tuning in both story diversity and adherence to narrative norms, thereby validating the efficacy of reinforcement learning for subjective generative tasks grounded in literary theory.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Reinforcement Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Despite the subjective nature of storytelling, past works on automatic story generation (ASG) have relied on limited ground truths for training and evaluation. In this work, we explore reinforcement learning (d-RLAIF) as a post-training alternative to supervised fine-tuning (SFT). We first apply Todorov's Theory of Narrative Equilibrium to establish principles that define desirable ASG qualities. We prompt 7B and 14B LLM-as-judge models with our principles to test alignment with human annotators and provide reward signals during d-RLAIF. We use Gemini-3-Flash to evaluate the output of our post-trained models and compare them to human-written stories from the TimeTravel dataset. We show that d-RLAIF offers a viable alternative to supervised fine-tuning (SFT)--producing stories that are more diverse and aligned with human narrative conventions. Our paper demonstrates the promise of reinforcement learning for linguistically grounded post-training for subjective tasks such as ASG.

Problem

• automatic story generation
• narrative theory
• subjective evaluation
• reinforcement learning
• story diversity

Innovation

• reinforcement learning
• narrative theory
• LLM-as-judge
• post-training
• story generation