Learning to Negotiate via Voluntary Commitment

2025-03-05

AI Summary

Real-world multi-agent systems often operate in mixed-motive settings: while cooperation enhances social welfare, it frequently fails due to unenforceable commitments. To address this, we introduce Markov Commitment Games (MCGs), the first formal framework modeling voluntary, learnable commitments to future behavior. We design an incentive-compatible reinforcement learning paradigm that guarantees individual rationality while driving agents to converge spontaneously to high-social-welfare equilibria. Furthermore, we develop a policy-gradient-based learnable commitment protocol. Evaluated on multiple challenging mixed-motive tasks—including sequential social dilemmas and coordination benchmarks—our approach consistently outperforms state-of-the-art baselines, achieving faster convergence and higher average returns. This work establishes a scalable, trainable foundation—both theoretical and algorithmic—for fostering trustworthy, decentralized cooperation among autonomous agents.

Technology Category

• Multiagent Systems: Mechanism Design
• Game Theory and Economic Paradigms: Cooperative Game Theory
• Machine Learning: Reinforcement Learning

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The partial alignment and conflict of autonomous agents lead to mixed-motive scenarios in many real-world applications. However, agents may fail to cooperate in practice even when cooperation yields a better outcome. One well known reason for this failure comes from non-credible commitments. To facilitate commitments among agents for better cooperation, we define Markov Commitment Games (MCGs), a variant of commitment games, where agents can voluntarily commit to their proposed future plans. Based on MCGs, we propose a learnable commitment protocol via policy gradients. We further propose incentive-compatible learning to accelerate convergence to equilibria with better social welfare. Experimental results in challenging mixed-motive tasks demonstrate faster empirical convergence and higher returns for our method compared with its counterparts. Our code is available at https://github.com/shuhui-zhu/DCL.

Problem

• Addresses non-credible commitments in autonomous agent cooperation.
• Introduces Markov Commitment Games for voluntary agent commitments.
• Proposes incentive-compatible learning to improve social welfare outcomes.

Innovation

• Markov Commitment Games for voluntary commitments
• Learnable commitment protocol via policy gradients
• Incentive-compatible learning for faster convergence