Are You There God? Lightweight Narrative Annotation of Christian Fiction with LMs

2025-07-25

AI Summary

This study addresses the longstanding absence of systematic computational research on Christian fiction by introducing, for the first time, a fine-grained annotation framework for miracle narratives (“divine agency”) and proposing a lightweight language model fine-tuning approach that achieves near-human annotation accuracy (F1 = 0.92) on resource-constrained hardware. The methodology integrates expert annotation, large-language-model-assisted instruction transfer, narrative classification, and topic modeling to enable scalable cross-text analysis. Key findings include: (1) the left-wing “Behind” series exhibits statistically significant divergence from the broader genre in its representation of divine action; and (2) robust gender-based differences emerge between male and female authors in miracle frequency, agent specification (e.g., God vs. intermediaries), and modes of divine intervention. The work establishes a reproducible methodological paradigm and empirical foundation for computational humanities research on religious literature.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In addition to its more widely studied political activities, the American Evangelical movement has a well-developed but less externally visible cultural and literary side. Christian Fiction, however, has been little studied, and what scholarly attention there is has focused on the explosively popular Left Behind series. In this work, we use computational tools to provide both a broad topical overview of Christian Fiction as a genre and a more directed exploration of how its authors depict divine acts. Working with human annotators we first developed definitions and a codebook for "acts of God." We then adapted those instructions designed for human annotators for use by a recent, lightweight LM with the assistance of a much larger model. The laptop-scale LM is capable of matching human annotations, even when the task is subtle and challenging. Using these annotations, we show that significant and meaningful differences exist between the Left Behind books and Christian Fiction more broadly and between books by male and female authors.

Problem

• Analyzing divine acts depiction in Christian Fiction
• Comparing Left Behind series with broader Christian Fiction
• Exploring gender differences in Christian Fiction authorship

Innovation

• Lightweight LM for narrative annotation
• Human-LM collaboration for codebook adaptation
• Laptop-scale LM matches human annotation accuracy