Explaining Decisions of Agents in Mixed-Motive Games

2024-07-21

arXiv.org

AI Summary

In mixed-motive games—where cooperation and competition coexist—agent decisions remain largely uninterpretable, posing a critical challenge for explainable AI (XAI) in strategic multi-agent settings. Method: This paper introduces the first systematic XAI framework for such games, unifying three key interaction mechanisms: cheap talk, strategic competition, and implicit action-based communication. It integrates game-theoretic modeling, counterfactual reasoning, natural language generation, and behavioral attribution to enable intent decomposition and attribution in non-zero-sum multi-agent environments. Contribution/Results: The framework demonstrates cross-game generalization, validated on seven-player no-communication Diplomacy and three-player language-based Prisoner’s Dilemma. Human subjects achieved 28.6% higher accuracy in interpreting agent intentions, with statistically significant gains in trust (p < 0.01). It overcomes a fundamental explanatory gap in existing XAI approaches for strategic multi-agent interactions.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Multiagent Systems: Mechanism Design
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

In recent years, agents have become capable of communicating seamlessly via natural language and navigating in environments that involve cooperation and competition, a fact that can introduce social dilemmas. Due to the interleaving of cooperation and competition, understanding agents' decision-making in such environments is challenging, and humans can benefit from obtaining explanations. However, such environments and scenarios have rarely been explored in the context of explainable AI. While some explanation methods for cooperative environments can be applied in mixed-motive setups, they do not address inter-agent competition, cheap-talk, or implicit communication by actions. In this work, we design explanation methods to address these issues. Then, we proceed to establish generality and demonstrate the applicability of the methods to three games with vastly different properties. Lastly, we demonstrate the effectiveness and usefulness of the methods for humans in two mixed-motive games. The first is a challenging 7-player game called no-press Diplomacy. The second is a 3-player game inspired by the prisoner's dilemma, featuring communication in natural language.

Problem

• Explaining agent decisions in mixed-motive games
• Addressing inter-agent competition and communication challenges
• Developing methods for human-understandable AI explanations

Innovation

• Designing explanation methods for mixed-motive games
• Addressing inter-agent competition and communication issues
• Applying methods to Diplomacy and prisoner's dilemma