SAGE: A Hierarchical Framework for Evaluating Interpretive Literary Quality in Narratives

2026-09-06

AI Summary

本文提出SAGE框架，通过结合规则评估与大语言模型来评价叙事中的文学质量，解决了现有自然语言生成指标无法衡量文化、情感和哲学维度的问题。

Abstract

Assessing the literary quality of narratives requires evaluating interpretive dimensions (cultural representation, emotional depth, and philosophical engagement) that existing NLG metrics cannot measure. We introduce SAGE, a six-layer evaluation framework that separates rule-based assessment of observable textual properties from LLM-based evaluation of interpretive qualities drawn from cultural theory, affect theory, and existentialist philosophy. Each interpretive layer is assessed through multi-round iterative LLM evaluation with independent cross-validation, achieving measurement-grade reliability (98.8% convergence,>94% inter-rater agreement) stable across evaluator models. Validated on 600 evaluations across 100 short stories, our central finding is a systematic capability boundary: emotional-psychological representation approaches human levels, while cultural critique and philosophical depth exhibit approximately double the gap. LLM-generated narratives score below even commercial genre fiction on all three layers. We interpret this as a boundary between pattern-reproducible literary capacities learnable from training corpora and stance-requiring ones demanding cultural positioning and philosophical engagement that pattern matching alone cannot provide.

Problem

• narratives
• literary quality
• interpretive dimensions
• cultural representation
• emotional depth

Innovation

• Hierarchical Framework
• Interpretive Literary Quality
• LLM-based Evaluation
• Cultural Theory
• Affective Depth