Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference

2026-08-27

AI Summary

本文针对大型语言模型在模拟人物时的个性一致性问题，提出PRISM框架，通过结构化逆向推理任务来改进评估方法。

Abstract

As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an agent's behavior consistently reflects the psychological and stylistic characteristics of a target persona, has become a critical requirement. However, existing evaluation paradigms primarily rely on either holistic LLM-based judges, which are prone to "holistic appraisal hallucination'', or static psychometric inventories, which fail to capture the context-dependent fidelity required in dynamic dialogue. To address these limitations, we propose PRISM (Persona Reasoning with Inverse SFL-based Modeling), a psycholinguistically grounded framework that reformulates persona fidelity evaluation as a structured inverse inference task. Inspired by Systemic Functional Linguistics (SFL), PRISM decomposes persona fidelity into three functional dimensions: Task Framing, Interpersonal Stance, and Linguistic Style. It estimates dimension-specific evidence over a persona-conditioned label space and aggregates these signals into an interpretable and auditable evaluation process. Experiments show that PRISM yields more accurate and stable judgements than traditional holistic judging, providing a more reliable framework for persona fidelity evaluation.

Problem

• persona fidelity
• large language models
• dynamic dialogue

Innovation

• PRISM
• Persona Fidelity
• Inverse SFL-based Modeling
• Systemic Functional Linguistics