Annotating References to Mythological Entities in French Literature

2024-12-24

AI Summary

This study addresses the systematic identification and interpretation of Greco-Roman mythological references in modern and contemporary French literary texts—a task requiring fine-grained semantic analysis and domain-specific hermeneutic expertise. Method: We propose the first fine-grained annotation schema for literary mythological entities and develop a hybrid annotation framework integrating customized prompt engineering, large language models (LLMs; exemplified by ChatGPT), and rigorous human verification. Contribution/Results: We empirically establish LLMs’ capabilities and limitations in jointly performing mythological reference identification and literary interpretation: achieving high-accuracy annotation and insightful interpretive generation, while quantitatively revealing significant hallucination risks—particularly in locating mythological elements within long literary passages. Based on these findings, we advocate a cautious human-in-the-loop annotation paradigm, offering both methodological guidance and practical implementation pathways for fine-grained AI-assisted research in the humanities.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Application Domains: Humanities & Computational Social Science
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In this paper, we explore the relevance of large language models (LLMs) for annotating references to Roman and Greek mythological entities in modern and contemporary French literature. We present an annotation scheme and demonstrate that recent LLMs can be directly applied to follow this scheme effectively, although not without occasionally making significant analytical errors. Additionally, we show that LLMs (and, more specifically, ChatGPT) are capable of offering interpretative insights into the use of mythological references by literary authors. However, we also find that LLMs struggle to accurately identify relevant passages in novels (when used as an information retrieval engine), often hallucinating and generating fabricated examples-an issue that raises significant ethical concerns. Nonetheless, when used carefully, LLMs remain valuable tools for performing annotations with high accuracy, especially for tasks that would be difficult to annotate comprehensively on a large scale through manual methods alone.

Problem

• French literature
• mythology
• textual analysis

Innovation

• Large Language Models
• Mythological References
• Literary Analysis