Benevolent Dictators? On LLM Agent Behavior in Dictator Games

2025-11-11

AI Summary

Prior studies evaluating LLM agents in the Dictator Game overlook the critical influence of system prompts and fail to systematically assess result sensitivity to prompt variations. Method: We propose LLM-ABS, a framework that constructs robust behavioral baselines via neutral system prompt variants and uncovers decision logic through linguistic feature analysis—integrating prompt engineering, controlled-variable experiments, and NLP-based semantic parsing. Results: (1) LLMs exhibit strong fairness preferences; (2) system prompts significantly modulate fairness versus self-interest tendencies, with minor modifications inducing measurable behavioral shifts; (3) distinct language-level expression patterns emerge across models under open-ended instructions. This work provides the first systematic validation of prompt sensitivity in social preference evaluation and establishes a reliable, neutrality-grounded assessment paradigm—laying a methodological foundation for modeling AI social preferences.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Multiagent Systems: Mechanism Design
• Knowledge Representation and Reasoning: Preferences

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In behavioral sciences, experiments such as the ultimatum game are conducted to assess preferences for fairness or self-interest of study participants. In the dictator game, a simplified version of the ultimatum game where only one of two players makes a single decision, the dictator unilaterally decides how to split a fixed sum of money between themselves and the other player. Although recent studies have explored behavioral patterns of AI agents based on Large Language Models (LLMs) instructed to adopt different personas, we question the robustness of these results. In particular, many of these studies overlook the role of the system prompt - the underlying instructions that shape the model's behavior - and do not account for how sensitive results can be to slight changes in prompts. However, a robust baseline is essential when studying highly complex behavioral aspects of LLMs. To overcome previous limitations, we propose the LLM agent behavior study (LLM-ABS) framework to (i) explore how different system prompts influence model behavior, (ii) get more reliable insights into agent preferences by using neutral prompt variations, and (iii) analyze linguistic features in responses to open-ended instructions by LLM agents to better understand the reasoning behind their behavior. We found that agents often exhibit a strong preference for fairness, as well as a significant impact of the system prompt on their behavior. From a linguistic perspective, we identify that models express their responses differently. Although prompt sensitivity remains a persistent challenge, our proposed framework demonstrates a robust foundation for LLM agent behavior studies. Our code artifacts are available at https://github.com/andreaseinwiller/LLM-ABS.

Problem

• Assessing LLM agent fairness preferences in dictator games
• Examining system prompt sensitivity on agent behavior
• Developing robust framework for LLM behavioral analysis

Innovation

• Proposed LLM-ABS framework for agent behavior studies
• Used neutral prompt variations for reliable insights
• Analyzed linguistic features to understand reasoning