LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writers

2026-02-17

AI Summary

This study addresses the tendency of large language models (LLMs) to produce bland, clichéd creative writing due to excessive determinism, lacking the uncertainty and literary richness characteristic of human authorship. For the first time, it systematically quantifies the “uncertainty gap” between human writers and 28 LLMs on a high-quality story dataset, integrating information-theoretic measures, controlled generation experiments, and comparative analyses of instruction-tuned and reasoning-based models. The findings reveal that human writing exhibits significantly higher uncertainty than model outputs, and that current alignment strategies—particularly instruction tuning—further widen this gap, especially in creative contexts where uncertainty strongly correlates with perceived writing quality. Building on these insights, the work proposes a novel uncertainty-aware alignment paradigm to foster more literarily expressive generation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We argue that uncertainty is a key and understudied limitation of LLMs' performance in creative writing, which is often characterized as trite and cliché-ridden. Literary theory identifies uncertainty as a necessary condition for creative expression, while current alignment strategies steer models away from uncertain outputs to ensure factuality and reduce hallucination. We formalize this tension by quantifying the "uncertainty gap" between human-authored stories and model-generated continuations. Through a controlled information-theoretic analysis of 28 LLMs on high-quality storytelling datasets, we demonstrate that human writing consistently exhibits significantly higher uncertainty than model outputs. We find that instruction-tuned and reasoning models exacerbate this trend compared to their base counterparts; furthermore, the gap is more pronounced in creative writing than in functional domains, and strongly correlates to writing quality. Achieving human-level creativity requires new uncertainty-aware alignment paradigms that can distinguish between destructive hallucinations and the constructive ambiguity required for literary richness.

Problem

• uncertainty
• creative writing
• large language models
• hallucination
• literary creativity

Innovation

• uncertainty gap
• creative writing
• large language models
• information-theoretic analysis
• alignment paradigms