Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games

2024-10-10

arXiv.org

AI Summary

In mixed-motive multi-agent games, balancing altruistic cooperation with self-preservation remains challenging. To address this, we propose LASE, a distributed reinforcement learning algorithm grounded in empathic reasoning. Its key contributions are: (1) dynamic social relationship estimation via counterfactual inference, enabling adaptive regulation of gift-giving behavior; (2) a perspective-sampling module that models the distribution of others’ policies, jointly optimizing altruism and robustness; and (3) a dynamic reward allocation mechanism that promotes fair collaboration. Evaluated across spatiotemporally extended multi-type game settings—including cooperators, defectors, and imitators—LASE significantly improves collective cooperation levels and policy robustness while ensuring interaction fairness and adaptability.

Technology Category

• Multiagent Systems: Mechanism Design
• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Real-world multi-agent scenarios often involve mixed motives, demanding altruistic agents capable of self-protection against potential exploitation. However, existing approaches often struggle to achieve both objectives. In this paper, based on that empathic responses are modulated by inferred social relationships between agents, we propose LASE Learning to balance Altruism and Self-interest based on Empathy), a distributed multi-agent reinforcement learning algorithm that fosters altruistic cooperation through gifting while avoiding exploitation by other agents in mixed-motive games. LASE allocates a portion of its rewards to co-players as gifts, with this allocation adapting dynamically based on the social relationship -- a metric evaluating the friendliness of co-players estimated by counterfactual reasoning. In particular, social relationship measures each co-player by comparing the estimated $Q$-function of current joint action to a counterfactual baseline which marginalizes the co-player's action, with its action distribution inferred by a perspective-taking module. Comprehensive experiments are performed in spatially and temporally extended mixed-motive games, demonstrating LASE's ability to promote group collaboration without compromising fairness and its capacity to adapt policies to various types of interactive co-players.

Problem

• Multi-agent Scenarios
• Balancing Altruism and Self-protection
• Bullying Prevention

Innovation

• LASE Learning Algorithm
• Multi-Agent Cooperation
• Fairness and Self-Protection