LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game

2025-10-06

AI Summary

This study investigates the Theory of Mind (ToM) capabilities of large language models (LLMs) in incomplete-information cooperative games, specifically examining whether modeling partner intentions—rather than higher-order reasoning—is more critical for effective collaboration. Method: We introduce Hanabi-ToM, the first systematic ToM benchmark for dynamic multi-agent cooperation, built upon the cooperative card game Hanabi. It integrates LLM-driven agent simulation with an automated evaluation framework to quantify correlations between ToM depth (first-order vs. second-order) and gameplay performance. Contribution/Results: Empirical results demonstrate that first-order ToM—inferring others’ beliefs and intentions—significantly improves collaborative success, outperforming second-order ToM; ToM proficiency exhibits a strong positive correlation with game scores. This work provides the first empirical evidence that low-order mental modeling is pivotal for cooperative AI, establishing a novel benchmark and theoretical foundation for designing interpretable, trustworthy multi-agent systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Multiagent Systems: Mechanism Design
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Effective multi-agent collaboration requires agents to infer the rationale behind others' actions, a capability rooted in Theory-of-Mind (ToM). While recent Large Language Models (LLMs) excel at logical inference, their ability to infer rationale in dynamic, collaborative settings remains under-explored. This study introduces LLM-Hanabi, a novel benchmark that uses the cooperative game Hanabi to evaluate the rationale inference and ToM of LLMs. Our framework features an automated evaluation system that measures both game performance and ToM proficiency. Across a range of models, we find a significant positive correlation between ToM and in-game success. Notably, first-order ToM (interpreting others' intent) correlates more strongly with performance than second-order ToM (predicting others' interpretations). These findings highlight that for effective AI collaboration, the ability to accurately interpret a partner's rationale is more critical than higher-order reasoning. We conclude that prioritizing first-order ToM is a promising direction for enhancing the collaborative capabilities of future models.

Problem

• Evaluating LLMs' rationale inference in multi-agent collaboration games
• Assessing Theory-of-Mind capabilities in imperfect information settings
• Measuring correlation between ToM proficiency and collaborative performance

Innovation

• Uses Hanabi game for evaluating multi-agent collaboration
• Automated system measures game performance and Theory-of-Mind
• Prioritizes first-order Theory-of-Mind over higher-order reasoning