Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment

2025-06-06

AI Summary

Can large language models (LLMs) appropriately express personality traits within diverse cultural contexts? This paper introduces CulturalPersonas, the first cross-cultural benchmark for evaluating personality expression—comprising 3,000 scenario-based questions grounded in indigenous values across six countries. Methodologically, it pioneers a systematic integration of Hofstede’s cultural dimensions with the Five-Factor Model of personality, enabling a scenario-driven evaluation framework featuring both multiple-choice and open-ended responses. Personality distribution alignment is quantified via the Wasserstein distance, and cultural sensitivity is rigorously validated through multi-national human annotation. Experiments across three major LLM families demonstrate that our approach improves personality distribution alignment by over 20% (statistically significant reduction in Wasserstein distance), markedly enhancing cultural coherence and personality expressiveness of model outputs.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

As LLMs become central to interactive applications, ranging from tutoring to mental health, the ability to express personality in culturally appropriate ways is increasingly important. While recent works have explored personality evaluation of LLMs, they largely overlook the interplay between culture and personality. To address this, we introduce CulturalPersonas, the first large-scale benchmark with human validation for evaluating LLMs' personality expression in culturally grounded, behaviorally rich contexts. Our dataset spans 3,000 scenario-based questions across six diverse countries, designed to elicit personality through everyday scenarios rooted in local values. We evaluate three LLMs, using both multiple-choice and open-ended response formats. Our results show that CulturalPersonas improves alignment with country-specific human personality distributions (over a 20% reduction in Wasserstein distance across models and countries) and elicits more expressive, culturally coherent outputs compared to existing benchmarks. CulturalPersonas surfaces meaningful modulated trait outputs in response to culturally grounded prompts, offering new directions for aligning LLMs to global norms of behavior. By bridging personality expression and cultural nuance, we envision that CulturalPersonas will pave the way for more socially intelligent and globally adaptive LLMs.

Problem

• Evaluating LLMs' personality expression across diverse cultures
• Assessing cultural appropriateness in LLM personality outputs
• Bridging cultural nuances with personality traits in LLMs

Innovation

• Introduces CulturalPersonas benchmark for LLM personality evaluation
• Uses 3000 scenario-based questions across six diverse countries
• Reduces Wasserstein distance by 20% for trait alignment