Generation, Evaluation, and Explanation of Novelists'Styles with Single-Token Prompts

2025-11-25

AI Summary

Addressing the dual challenges in stylometry—scarce paired training data and heavy reliance on manual evaluation—this paper proposes a novel single-token prompt-based fine-tuning method that enables controllable generation of 19th-century literary styles (e.g., Dickens, Austen) without parallel corpora. We further design a Transformer-based style discriminator augmented with attention mechanisms and gradient-based interpretability techniques (e.g., integrated gradients), enabling automated, attribution-aware style assessment. Our framework integrates syntactic contrastive analysis to jointly model the full pipeline—from style-conditioned generation to interpretable evaluation. Experiments demonstrate that generated texts closely align with target authors’ linguistic patterns; moreover, automated evaluations exhibit strong agreement with human expert judgments (Cohen’s κ = 0.82), confirming the framework’s reliability and generalizability. All code and models are publicly released.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Recent advances in large language models have created new opportunities for stylometry, the study of writing styles and authorship. Two challenges, however, remain central: training generative models when no paired data exist, and evaluating stylistic text without relying only on human judgment. In this work, we present a framework for both generating and evaluating sentences in the style of 19th-century novelists. Large language models are fine-tuned with minimal, single-token prompts to produce text in the voices of authors such as Dickens, Austen, Twain, Alcott, and Melville. To assess these generative models, we employ a transformer-based detector trained on authentic sentences, using it both as a classifier and as a tool for stylistic explanation. We complement this with syntactic comparisons and explainable AI methods, including attention-based and gradient-based analyses, to identify the linguistic cues that drive stylistic imitation. Our findings show that the generated text reflects the authors'distinctive patterns and that AI-based evaluation offers a reliable alternative to human assessment. All artifacts of this work are published online.

Problem

• Generating 19th-century novelist styles using single-token prompts
• Evaluating stylistic imitation without relying on human judgment
• Explaining linguistic cues driving authorship imitation through AI

Innovation

• Fine-tuning LLMs with single-token prompts
• Using transformer-based detector for evaluation
• Applying explainable AI methods for stylistic analysis