Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment

2025-04-15

AI Summary

This study investigates the alignment between large language models’ (LLMs’) moral decisions—elicited via socially grounded personas (e.g., political orientation, gender, occupation)—and human moral judgments in ethical dilemmas. Methodologically, it employs prompt-engineered, multidimensional persona scaffolding integrated with the Moral Machine experimental paradigm and rigorous statistical significance testing to conduct cross-demographic alignment evaluation. The work provides the first empirical evidence that LLMs exhibit strong persona-dependent moral reasoning: political orientation predominantly governs both the direction and magnitude of bias, manifesting a “partisan sorting” effect; moreover, decision variability across personas significantly exceeds human inter-individual variability in critical judgment tasks. These findings expose systemic risks in deploying LLMs for real-world ethical decision-making and offer novel empirical evidence and methodological insights for developing trustworthy, human-aligned AI systems.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Deploying large language models (LLMs) with agency in real-world applications raises critical questions about how these models will behave. In particular, how will their decisions align with humans when faced with moral dilemmas? This study examines the alignment between LLM-driven decisions and human judgment in various contexts of the moral machine experiment, including personas reflecting different sociodemographics. We find that the moral decisions of LLMs vary substantially by persona, showing greater shifts in moral decisions for critical tasks than humans. Our data also indicate an interesting partisan sorting phenomenon, where political persona predominates the direction and degree of LLM decisions. We discuss the ethical implications and risks associated with deploying these models in applications that involve moral decisions.

Problem

• Examining LLM-human alignment in moral dilemmas
• Assessing persona-dependent moral decision shifts in LLMs
• Investigating political persona influence on LLM choices

Innovation

• Persona-dependent LLM alignment for moral decisions
• Examining sociodemographic influences on LLM choices
• Identifying partisan sorting in LLM moral behavior