WereWolf-Plus: An Update of Werewolf Game setting Based on DSGBench

2025-06-15

AI Summary

Existing Werewolf benchmark platforms suffer from oversimplified game settings, unidimensional evaluation metrics, and poor extensibility. To address these limitations, we propose WolF-Bench—a novel open-source evaluation platform for multi-agent strategic reasoning. Methodologically, it introduces the first “multi-model–multi-dimension–multi-method” evaluation framework; supports customizable role configuration (e.g., Seer, Witch, Hunter) with role-specific reasoning enhancements; establishes a comprehensive quantitative metric suite covering all roles; and extends the DSGBench simulation framework by integrating LLM-based role agents, dynamic role assignment, collaborative behavior modeling, and quantification of social influence. Experimental results demonstrate that WolF-Bench significantly improves measurability and cross-model comparability of strategic reasoning, coordinated decision-making, and social interaction capabilities, while enabling flexible experimental configurations and fair, reproducible evaluation.

Technology Category

• Multiagent Systems: Adversarial Agents
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

With the rapid development of LLM-based agents, increasing attention has been given to their social interaction and strategic reasoning capabilities. However, existing Werewolf-based benchmarking platforms suffer from overly simplified game settings, incomplete evaluation metrics, and poor scalability. To address these limitations, we propose WereWolf-Plus, a multi-model, multi-dimensional, and multi-method benchmarking platform for evaluating multi-agent strategic reasoning in the Werewolf game. The platform offers strong extensibility, supporting customizable configurations for roles such as Seer, Witch, Hunter, Guard, and Sheriff, along with flexible model assignment and reasoning enhancement strategies for different roles. In addition, we introduce a comprehensive set of quantitative evaluation metrics for all special roles, werewolves, and the sheriff, and enrich the assessment dimensions for agent reasoning ability, cooperation capacity, and social influence. WereWolf-Plus provides a more flexible and reliable environment for advancing research on inference and strategic interaction within multi-agent communities. Our code is open sourced at https://github.com/MinstrelsyXia/WereWolfPlus.

Problem

• Overly simplified game settings in existing Werewolf benchmarks
• Incomplete evaluation metrics for multi-agent strategic reasoning
• Poor scalability in current Werewolf-based benchmarking platforms

Innovation

• Multi-model multi-method Werewolf benchmarking platform
• Customizable roles and flexible model assignment
• Quantitative metrics for reasoning and social influence