Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires

2025-02-07

AI Summary

This study investigates whether large language models (LLMs) exhibit measurable personality traits. Method: It introduces standardized psychological instruments—particularly the Big Five Inventory—into LLM evaluation for the first time, establishing a rigorously controlled “LLM Personality Profiling Framework” that mitigates data contamination. The framework integrates psychologically grounded prompt engineering, multi-turn response consistency analysis, cross-model controllable text generation, and statistical significance testing. Contribution/Results: Empirical results demonstrate that mainstream LLMs exhibit stable, replicable, and discriminable personality profiles—for instance, Llama-series models show significantly higher openness and lower neuroticism, whereas GPT-series models display higher conscientiousness and moderate extraversion. This work establishes a novel, quantitatively grounded paradigm for personality assessment of LLMs, advancing model behavioral interpretability and informing human-AI interaction design with empirical psychological foundations.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Psychological assessment tools have long helped humans understand behavioural patterns. While Large Language Models (LLMs) can generate content comparable to that of humans, we explore whether they exhibit personality traits. To this end, this work applies psychological tools to LLMs in diverse scenarios to generate personality profiles. Using established trait-based questionnaires such as the Big Five Inventory and by addressing the possibility of training data contamination, we examine the dimensional variability and dominance of LLMs across five core personality dimensions: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Our findings reveal that LLMs exhibit unique dominant traits, varying characteristics, and distinct personality profiles even within the same family of models.

Problem

• Assessing personality traits in LLMs
• Using psychological tools for LLM evaluation
• Exploring variability in LLM personality dimensions

Innovation

• Applies psychological questionnaires to LLMs
• Examines Big Five personality dimensions
• Assesses dimensional variability in models