SOTOPIA-TOM: Evaluating Information Management in Multi-Agent Interaction with Theory of Mind

2026-05-04

AI Summary

This study addresses the challenge that large language model agents struggle to manage information asymmetry in multi-party interactions and lack standardized benchmarks for evaluating their disclosure and privacy-preserving capabilities. To this end, the authors introduce a multi-agent environment supporting both public and private communication, encompassing 160 human-reviewed scenarios across eight industry domains, along with INFOMGMT—the first evaluation framework for multidimensional information management. They further propose a novel Theory of Mind (ToM)-based intervention strategy, ToM-Coach, which significantly improves agents’ ability to balance collaborative efficiency and privacy protection. Experiments show that ToM-Coach reduces GPT-4o’s critical privacy violation rate from 9.9% to 2.2% and increases its INFOMGMT score from 15% to 40%, while even the strongest current model, GPT-5, achieves only 62%.

Abstract

As LLM-based agents are increasingly interacting in multi-party settings, they need to properly handle information asymmetry, i.e., knowing when and to whom to disclose information is appropriate. Yet, existing benchmarks fail to measure this ability in realistic multi-party settings. Thus, we introduce SOTOPIA-TOM, a multi-dimensional benchmarking framework to evaluate LLM agents' ability to successfully navigate information asymmetric and privacy sensitive multi-party interactions. We create an interaction environment which enables both public (broadcast) and private (direct message) communication, and craft 160 human-reviewed scenarios across eight industry sectors, each involving 3 to 5 agents with partitioned private knowledge and channel-dependent sharing policies. To measure interaction abilities, we create a multi-dimensional evaluation framework to assess how well agents share useful information, seek missing details, coordinate efficiently, and protect privacy, which we also combine into a composite INFOMGMT metric. Results show that, across 6 LLM backbones and prompting strategies (vanilla, CoT-privacy, and ToM-based interventions), even the largest high-reasoning model (GPT-5) reaches only a 62% INFOMGMT score, which indicates persistent deficiencies in information seeking and privacy-aware decision-making. Additionally, ToM-based interventions more consistently improve the overall coordination-privacy balance (for example, relative to the vanilla baseline, ToM-Coach reduces critical privacy violations on GPT-4o from 9.9% to 2.2% while increasing the composite InfoMgmt score more than 2.5x from 15% to 40%). Overall, SOTOPIA-TOM exposes persistent limitations of current LLM agents in complex, information-asymmetric coordination and provides an extensible testbed for developing more privacy-aware, theory-of-mind capable multi-agent systems.

Problem

• information asymmetry
• multi-agent interaction
• privacy-sensitive communication
• theory of mind
• information management

Innovation

• Theory of Mind
• Information Asymmetry
• Multi-Agent Interaction
• Privacy-Aware Coordination
• Benchmarking Framework