CLIN: an Objective Framework for Evaluating Creativity in Short Persian Literary Text

2026-08-31

AI Summary

本文针对波斯语文本的创造性评估难题，提出CLIN框架，利用新颖性、词汇聚类和多样性分别衡量原创性、流畅性和细节性，以低成本实现与人类判断相当的一致性。

Abstract

Evaluating creativity in large language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine how reliably LLMs evaluate short literary text in Persian, a low-resource language, across multiple evaluation strategies and prompt formulations. We find that LLM-human agreement varies substantially across dimensions: alignment is stronger for structured TTCT-derived properties such as Originality, Fluency, and Elaboration, but considerably weaker for more subjective dimensions, particularly Emotion and Attractiveness. Judgments are also sensitive to prompt formulation, while few-shot prompting, ensembling, and multi-agent debate provide no consistent improvement. Motivated by this dimension-dependent behavior, we investigate whether structured creativity dimensions can instead be approximated using simple, interpretable proxy metrics. We introduce CLIN, which evaluates three TTCT-derived dimensions separately using topic-aware novelty for Originality, contextual lexical clustering for Fluency, and lexical diversity for Elaboration. These proxies achieve human alignment comparable to or better than the strongest zero-shot LLM judge in our setting while requiring substantially lower evaluation cost.

Problem

• creativity
• large language model
• Persian
• evaluation
• multidimensional

Innovation

• CLIN
• Topic-aware Novelty
• Contextual Lexical Clustering
• Lexical Diversity