Do Large Language Models Understand Morality Across Cultures?

2025-07-28

AI Summary

This study investigates large language models’ (LLMs) capacity to comprehend cross-cultural moral concepts. Addressing the lack of systematic evaluation in prior work, we propose a three-pronged validation framework: (1) variance-based analysis of moral score disparities across cultures; (2) clustering alignment to assess consistency between model outputs and empirically grounded cultural structures from the World Values Survey (WVS) and European Social Survey (ESS); and (3) targeted probe prompting using predefined moral word pairs. Experiments span major open- and closed-weight LLMs under diverse cultural scenarios, quantifying moral judgment biases. Results reveal pervasive cross-cultural moral compression—i.e., reduced intercultural variation—in current LLMs, coupled with significantly low alignment to real-world survey data, exposing latent cultural biases. To our knowledge, this is the first systematic demonstration of cultural misalignment in LLM moral modeling, providing a reproducible evaluation framework and empirical foundation for advancing culturally equitable AI.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Morality & Value-based AI
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advancements in large language models (LLMs) have established them as powerful tools across numerous domains. However, persistent concerns about embedded biases, such as gender, racial, and cultural biases arising from their training data, raise significant questions about the ethical use and societal consequences of these technologies. This study investigates the extent to which LLMs capture cross-cultural differences and similarities in moral perspectives. Specifically, we examine whether LLM outputs align with patterns observed in international survey data on moral attitudes. To this end, we employ three complementary methods: (1) comparing variances in moral scores produced by models versus those reported in surveys, (2) conducting cluster alignment analyses to assess correspondence between country groupings derived from LLM outputs and survey data, and (3) directly probing models with comparative prompts using systematically chosen token pairs. Our results reveal that current LLMs often fail to reproduce the full spectrum of cross-cultural moral variation, tending to compress differences and exhibit low alignment with empirical survey patterns. These findings highlight a pressing need for more robust approaches to mitigate biases and improve cultural representativeness in LLMs. We conclude by discussing the implications for the responsible development and global deployment of LLMs, emphasizing fairness and ethical alignment.

Problem

• Assessing LLMs' understanding of cross-cultural moral perspectives
• Evaluating alignment between LLM outputs and international moral survey data
• Identifying biases and improving cultural representativeness in LLMs

Innovation

• Compare moral score variances between models and surveys
• Conduct cluster alignment analyses for country groupings
• Probe models with comparative prompts using token pairs