Collective cooperation without individual fidelity in LLM agents

2026-06-29

AI Summary

This study investigates whether large language model (LLM) agents can authentically replicate human cooperative decision-making in social simulations. By implementing an online prisoner’s dilemma experiment with identical interaction protocols, payoff structures, and network topologies used in human studies, the authors compare behavioral data from nine open-source LLM agents against empirical human data. The results show that LLMs successfully reproduce macro-level dynamics—such as an initial decline followed by stabilization in cooperation rates—but exhibit significant deviations at the micro level, particularly in individual heterogeneity and conditional cooperation strategies. Introducing stochasticity into agent behavior only partially improves micro-level alignment and fails to bridge fundamental discrepancies in underlying decision rules. These findings underscore the necessity of multidimensional validation when employing LLMs as proxies for human behavior in social science research.

Abstract

Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making. Here we test LLM agents against a direct empirical benchmark: a large-scale networked Prisoner's Dilemma experiment with human participants. Using the same interaction protocol, payoff structure, and network topologies, we compare nine open-weight LLMs with the human data. The selected model reproduces several macro-level features of cooperation dynamics, including the early decline and later stabilization of cooperation. This aggregate agreement, however, does not extend uniformly to finer levels of behavior. LLM populations underestimate individual-level heterogeneity and generate conditional cooperation patterns that differ from those observed in humans. Adding a fraction of random agents improves some aspects of micro-level agreement, but does not remove the mismatch in decision rules. These findings reveal a macro--micro dissociation in LLM-based social agents: collective outcomes can appear human-like even when the underlying behavioral distributions and mechanisms are not. They suggest that validating LLM agents as human surrogates requires comparisons across aggregate dynamics, individual heterogeneity, and context-dependent decision rules, rather than outcome-level agreement alone.

Problem

• LLM agents
• human decision-making
• cooperation dynamics
• individual heterogeneity
• behavioral fidelity

Innovation

• macro-micro dissociation
• LLM agents
• collective cooperation
• conditional cooperation
• behavioral validation