Strategic Bargaining in Multi-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiations

2026-07-07

AI Summary

This work addresses the challenge of suboptimal outcomes in finite-round concurrent negotiations between a single seller and multiple buyers, where existing large language models (LLMs) often fail due to a lack of economic rationality. To overcome this limitation, the authors propose a Reinforcement Learning with Verifiable Rewards (RLVR) framework that employs an objective, economics-driven reward function to train LLMs to balance exploration of high-valuation buyers against strategic surplus extraction. By incorporating verifiable economic outcomes into the training process—a first for LLM-based negotiation—this approach enables agents to intrinsically develop sophisticated behaviors such as price anchoring and strategic probing. The trained seller agent demonstrates strong generalization across unseen buyer strategies and budget distributions, significantly outperforming state-of-the-art LLMs in real economic surplus capture, bargaining efficiency, and success rate in securing high-value deals.

Abstract

Negotiation is a fundamental strategic interaction in management science, characterized by agents attempting to reach agreements while protecting private information, such as reservation costs and hidden valuations. A prevalent yet complex scenario involves a single seller negotiating concurrently with multiple buyers, each possessing heterogeneous, private budgets. In such settings, constrained by a limited number of communication turns, the seller must balance exploring the broader market to discover the highest valuation with concentrating sufficient turns on a single target buyer to secure the best possible outcome. Our analysis reveals a significant gap in standard Large Language Models (LLMs): while these models are linguistically proficient, they fail to act as effective economic decision-makers. Specifically, they exhibit a failure to explore the buyer pool, often fixating on the current highest bid rather than strategically investigating the market to discover latent high valuations. In this paper, we propose a specialized training recipe using Reinforcement Learning from Verifiable Rewards (RLVR). By anchoring the reward function to objective economic outcomes, the strategic balance between market discovery and surplus extraction emerges natively through the learning process. Our results demonstrate that the trained seller undergoes a multi-stage strategic evolution, learning to leverage price anchoring and strategic probing to identify more profitable counterparties. The agent extracts a substantially higher surplus than frontier models by both improving its persuasive bargaining skills and consistently closing deals with high-value buyers. Finally, we show that our seller strategies generalize robustly to unseen buyer negotiation styles and budget distributions.

Problem

• strategic bargaining
• multi-buyer markets
• LLM negotiations
• private budgets
• market exploration

Innovation

• Reinforcement Learning from Verifiable Rewards
• LLM Negotiation
• Multi-Buyer Market
• Strategic Bargaining
• Surplus Extraction