Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma

2025-01-27

AI Summary

This study investigates the emergence mechanisms of cooperative behavior in large language models (LLMs) within social dilemmas—particularly the repeated prisoner’s dilemma—and examines their population-level evolutionary dynamics. Method: Leveraging prompt engineering, we elicit full iterative strategies from LLMs including GPT-4, Claude, and Llama; integrate these into evolutionary game-theoretic models; and conduct multi-agent population simulations. Contribution/Results: We provide the first systematic quantification of intrinsic preference biases toward cooperation versus defection across diverse LLMs. Results demonstrate that model architecture and pretraining data significantly shape strategic inclinations; stable cooperation can evolve in certain LLM populations; and LLM social behavior exhibits both predictability and contextual sensitivity. This work establishes a theoretical foundation and empirical basis for understanding the nature of AI cooperation and designing trustworthy intelligent assistants.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: (Large) Language Models
• Game Theory and Economic Paradigms: Cooperative Game Theory

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As autonomous agents become more prevalent, understanding their collective behaviour in strategic interactions is crucial. This study investigates the emergent cooperative tendencies of systems of Large Language Model (LLM) agents in a social dilemma. Unlike previous research where LLMs output individual actions, we prompt state-of-the-art LLMs to generate complete strategies for iterated Prisoner's Dilemma. Using evolutionary game theory, we simulate populations of agents with different strategic dispositions (aggressive, cooperative, or neutral) and observe their evolutionary dynamics. Our findings reveal that different LLMs exhibit distinct biases affecting the relative success of aggressive versus cooperative strategies. This research provides insights into the potential long-term behaviour of systems of deployed LLM-based autonomous agents and highlights the importance of carefully considering the strategic environments in which they operate.

Problem

• Collaborative Strategy Games
• Cooperative Behavior
• Personality Traits

Innovation

• Evolutionary Game Theory
• Advanced AI Agents
• Strategic Thinking