Re-evaluating Theory of Mind evaluation in large language models

2025-02-28

AI Summary

This paper addresses the contentious question of whether large language models (LLMs) possess theory of mind (ToM) capabilities. It identifies a core conceptual conflation in existing evaluations—between *behavioral matching* (i.e., reproducing human-like responses) and *computational mechanism matching* (i.e., implementing human-like reasoning processes)—exacerbated by systematic task design biases. Adopting a cognitive science perspective, the study conducts a critical meta-assessment and diagnostically reconstructs canonical ToM tasks to clarify foundational conceptual assumptions. It then proposes a novel evaluation paradigm that explicitly distinguishes behavioral outputs from underlying computational mechanisms. The work reveals structural limitations in current ToM benchmarks and establishes a theoretically grounded, interpretable framework for assessing social cognition in AI—offering both conceptual clarity and methodological guidance for future rigorous evaluation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The question of whether large language models (LLMs) possess Theory of Mind (ToM) -- often defined as the ability to reason about others' mental states -- has sparked significant scientific and public interest. However, the evidence as to whether LLMs possess ToM is mixed, and the recent growth in evaluations has not resulted in a convergence. Here, we take inspiration from cognitive science to re-evaluate the state of ToM evaluation in LLMs. We argue that a major reason for the disagreement on whether LLMs have ToM is a lack of clarity on whether models should be expected to match human behaviors, or the computations underlying those behaviors. We also highlight ways in which current evaluations may be deviating from"pure"measurements of ToM abilities, which also contributes to the confusion. We conclude by discussing several directions for future research, including the relationship between ToM and pragmatic communication, which could advance our understanding of artificial systems as well as human cognition.

Problem

• Assessing Theory of Mind in large language models
• Clarifying expectations for human-like behavior in models
• Improving evaluation methods for ToM in artificial systems

Innovation

• Re-evaluates Theory of Mind in LLMs
• Compares human behaviors and model computations
• Proposes future research on ToM and communication