Reproducibility Study of Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation

2025-02-22

AI Summary

This study systematically evaluates large language models’ (LLMs) behavioral spectrum—cooperative, competitive, and adversarial—in stakeholder negotiation, while examining ethical dimensions including accessibility, fairness, privacy, and environmental impact. Method: We introduce a no-communication baseline, Pareto frontier analysis, structured information-leakage detection, and fairness inequality metrics; integrate open-source models (1.5B–70B) with GPT-4o Mini; and combine game-theoretic modeling, response-format consistency validation, and structural quantitative analysis. Contribution/Results: Models exceeding 10B parameters achieve performance comparable to closed-source counterparts. Crucially, single-agent negotiation strategies match or surpass multi-agent protocols requiring explicit inter-agent communication across diverse scenarios—challenging the assumed necessity of multi-agent interaction. These findings establish a novel paradigm for lightweight, controllable, and equitable LLM-based negotiation systems, grounded in rigorous empirical and ethical evaluation.

Technology Category

• Multiagent Systems: Agreement, Argumentation & Negotiation
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This paper presents a reproducibility study and extension of"Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation."We validate the original findings using a range of open-weight models (1.5B-70B parameters) and GPT-4o Mini while introducing several novel contributions. We analyze the Pareto front of the games, propose a communication-free baseline to test whether successful negotiations are possible without agent interaction, evaluate recent small language models' performance, analyze structural information leakage in model responses, and implement an inequality metric to assess negotiation fairness. Our results demonstrate that smaller models (<10B parameters) struggle with format adherence and coherent responses, but larger open-weight models can approach proprietary model performance. Additionally, in many scenarios, single-agent approaches can achieve comparable results to multi-agent negotiations, challenging assumptions about the necessity of agent communication to perform well on the benchmark. This work also provides insights into the accessibility, fairness, environmental impact, and privacy considerations of LLM-based negotiation systems.

Problem

• Validate findings on LLM-stakeholder negotiation dynamics.
• Assess negotiation fairness using an inequality metric.
• Evaluate small models' performance in negotiation tasks.

Innovation

• Open-weight model validation
• Communication-free baseline proposal
• Inequality metric implementation