Understanding Cultural Alignment in Multilingual LLMs via Natural Debate Statements

2026-02-13

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

In this work we investigate the sociocultural values learned by large language models (LLMs). We introduce a novel open-access dataset, Sociocultural Statements, constructed from natural debate statements using a multi-step methodology. The dataset is synthetically labeled to enable the quantization of sociocultural norms and beliefs that LLMs exhibit in their responses to these statements, according to the Hofstede cultural dimensions. We verify the accuracy of synthetic labels using human quality control on a representative sample. We conduct a comparative analysis between two groups of LLMs developed in different countries (U.S. and China), and use as a comparative baseline patterns observed in human measurements. Using this new dataset and the analysis above, we found that culturally-distinct LLMs reflect the values and norms of the countries in which they were developed, highlighting their inability to adapt to the sociocultural backgrounds of their users.

Problem

• cultural alignment
• multilingual LLMs
• sociocultural values
• Hofstede dimensions
• cross-cultural adaptation

Innovation

• cultural alignment
• multilingual LLMs
• Sociocultural Statements dataset
• Hofstede cultural dimensions
• synthetic labeling