Large Language Models Reflect the Ideology of their Creators

2024-10-24

arXiv.org

AI Summary

This study challenges the assumptions of “technological neutrality” and “bias-free” design by revealing systematic geopolitical divergence in large language models’ (LLMs) moral judgments. Method: Leveraging cross-lingual prompt engineering, we generated large-scale personality descriptions of political figures across the UN’s six official languages; these were automatically annotated using Moral Foundations Theory (MFT) and subjected to statistical analysis. Contribution/Results: We provide the first empirical evidence of ideologically structured moral reasoning in LLMs: U.S.-developed models exhibit internal progressive-value tensions, whereas Chinese models display a strategic ideological split—emphasizing authority and loyalty in external-facing outputs while attenuating these foundations domestically. These findings demonstrate that LLMs’ normative outputs systematically reflect both developers’ geographic context and audience-targeting strategies, offering critical evidence for understanding value embedding in AI and informing transnational AI governance frameworks.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) are trained on vast amounts of data to generate natural language, enabling them to perform tasks like text summarization and question answering. These models have become popular in artificial intelligence (AI) assistants like ChatGPT and already play an influential role in how humans access information. However, the behavior of LLMs varies depending on their design, training, and use. In this paper, we prompt a diverse panel of popular LLMs to describe a large number of prominent personalities with political relevance, in all six official languages of the United Nations. By identifying and analyzing moral assessments reflected in their responses, we find normative differences between LLMs from different geopolitical regions, as well as between the responses of the same LLM when prompted in different languages. Among only models in the United States, we find that popularly hypothesized disparities in political views are reflected in significant normative differences related to progressive values. Among Chinese models, we characterize a division between internationally- and domestically-focused models. Our results show that the ideological stance of an LLM appears to reflect the worldview of its creators. This poses the risk of political instrumentalization and raises concerns around technological and regulatory efforts with the stated aim of making LLMs ideologically 'unbiased'.

Problem

• Bias
• Objectivity
• Political Influence

Innovation

• Large Language Models
• Moral Evaluation Bias
• Political Tool Potential