EvoEmo: Towards Evolved Emotional Policies for LLM Agents in Multi-Turn Negotiation

2025-09-04

AI Summary

Current LLM-based negotiation agents treat emotions as passive preference signals, lacking active emotion regulation capabilities and thus remaining vulnerable to strategic exploitation by opponents. To address this, we propose EmoNegotiate—a novel framework that formalizes dynamic emotional expression as a Markov Decision Process (MDP) and introduces Population-based Evolutionary Reinforcement Learning (P-ERL) to optimize emotion policies. P-ERL integrates chain-of-thought-guided emotional reasoning, multi-objective reward modeling, and genetic algorithm-driven co-evolution to enable adaptive emotional state transitions aligned with strategic negotiation intent. Evaluated on a multi-round procurement negotiation benchmark, EmoNegotiate significantly outperforms state-of-the-art baselines: achieving +23.6% higher negotiation success rate, −18.4% fewer negotiation rounds on average, and +31.2% greater buyer cost savings. These results empirically validate that functional emotion modeling is critical for enhancing the robustness and efficacy of LLM-based negotiation agents in adversarial, goal-directed interactions.

Technology Category

• Multiagent Systems: Agreement, Argumentation & Negotiation
• Humans and AI: Emotional Intelligence
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Recent research on Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) has demonstrated that agents can engage in extit{complex}, extit{multi-turn} negotiations, opening new avenues for agentic AI. However, existing LLM agents largely overlook the functional role of emotions in such negotiations, instead generating passive, preference-driven emotional responses that make them vulnerable to manipulation and strategic exploitation by adversarial counterparts. To address this gap, we present EvoEmo, an evolutionary reinforcement learning framework that optimizes dynamic emotional expression in negotiations. EvoEmo models emotional state transitions as a Markov Decision Process and employs population-based genetic optimization to evolve high-reward emotion policies across diverse negotiation scenarios. We further propose an evaluation framework with two baselines -- vanilla strategies and fixed-emotion strategies -- for benchmarking emotion-aware negotiation. Extensive experiments and ablation studies show that EvoEmo consistently outperforms both baselines, achieving higher success rates, higher efficiency, and increased buyer savings. This findings highlight the importance of adaptive emotional expression in enabling more effective LLM agents for multi-turn negotiation.

Problem

• Optimizing dynamic emotional expression in multi-turn LLM negotiations
• Addressing vulnerability from passive emotional responses in agents
• Evolving high-reward emotion policies across diverse negotiation scenarios

Innovation

• Evolutionary reinforcement learning optimizes emotional expression
• Models emotional transitions as Markov Decision Process
• Employs genetic optimization for high-reward emotion policies