The author is dead, but what if they never lived? A reception experiment on Czech AI- and human-authored poetry

2025-11-26

AI Summary

This study investigates Czech native speakers’ perceptual differentiation and aesthetic evaluation of AI-generated versus human-authored poetry, addressing a gap in AI content reception research for low-resource, morphologically complex Slavic languages. Method: Poetry conforming to Czech morphological constraints was generated using large language models; a controlled reading experiment, statistical analysis, and logistic regression modeling were employed to assess authorship attribution accuracy and multidimensional aesthetic ratings. Contribution/Results: Participants’ authorship identification accuracy was only 45.8%—statistically indistinguishable from chance. Overall aesthetic ratings of AI-generated poems did not significantly differ from those of human-authored poems—and even exceeded them on certain dimensions—yet scores dropped significantly when poems were misattributed to AI. These findings demonstrate that authorship beliefs systematically bias aesthetic judgment, challenging the pervasive cognitive bias that AI-generated content is inherently inferior. The study establishes a methodological framework and theoretical insights for cross-linguistic, non-English AI aesthetics research.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Large language models are increasingly capable of producing creative texts, yet most studies on AI-generated poetry focus on English -- a language that dominates training data. In this paper, we examine the perception of AI- and human-written Czech poetry. We ask if Czech native speakers are able to identify it and how they aesthetically judge it. Participants performed at chance level when guessing authorship (45.8% correct on average), indicating that Czech AI-generated poems were largely indistinguishable from human-written ones. Aesthetic evaluations revealed a strong authorship bias: when participants believed a poem was AI-generated, they rated it as less favorably, even though AI poems were in fact rated equally or more favorably than human ones on average. The logistic regression model uncovered that the more the people liked a poem, the less probable was that they accurately assign the authorship. Familiarity with poetry or literary background had no effect on recognition accuracy. Our findings show that AI can convincingly produce poetry even in a morphologically complex, low-resource (with respect of the training data of AI models) Slavic language such as Czech. The results suggest that readers'beliefs about authorship and the aesthetic evaluation of the poem are interconnected.

Problem

• Examining perception of AI- and human-authored Czech poetry
• Assessing if Czech speakers can identify authorship and judge aesthetics
• Investigating authorship bias in aesthetic evaluations of Czech poems

Innovation

• AI-generated Czech poetry indistinguishable from human
• Logistic regression model linking aesthetic preference to authorship misattribution
• Authorship bias affects aesthetic ratings regardless of literary background