BeliefNest: A Joint Action Simulator for Embodied Agents with Theory of Mind

2025-05-18

AI Summary

Embodied agents lack explicit Theory of Mind (ToM) modeling capabilities for open-domain collaborative tasks. Method: We propose BeliefNest, an open-source joint-action simulator built in Minecraft, featuring the first dynamic hierarchical nested belief modeling framework. It structurally represents self- and other-centered multi-order belief states as parseable graph models, directly mapped to LLM prompts to enable interpretable, evaluable ToM-driven decision-making. The approach integrates embodied simulation, hierarchical belief graphs, LLM prompt engineering, and a novel false-belief task evaluation protocol. Contribution/Results: Experiments demonstrate that BeliefNest accurately infers others’ beliefs and predicts belief-guided behavior. Quantitative evaluation on standardized false-belief tasks confirms that nested belief modeling significantly enhances multi-agent coordination performance, establishing a new benchmark for interpretable, ToM-aware embodied AI.

Technology Category

• Multiagent Systems: Modeling other Agents
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

This paper introduces an open-source simulator, BeliefNest, designed to enable embodied agents to perform collaborative tasks by leveraging Theory of Mind. BeliefNest dynamically and hierarchically constructs simulators within a Minecraft environment, allowing agents to explicitly represent nested belief states about themselves and others. This enables agent control in open-domain tasks that require Theory of Mind reasoning. The simulator provides a prompt generation mechanism based on each belief state, facilitating the design and evaluation of methods for agent control utilizing large language models (LLMs). We demonstrate through experiments that agents can infer others' beliefs and predict their belief-based actions in false-belief tasks.

Problem

• Simulate collaborative tasks for embodied agents using Theory of Mind
• Model nested belief states in open-domain tasks dynamically
• Enable belief-based action prediction in false-belief scenarios

Innovation

• Dynamic hierarchical simulators in Minecraft
• Explicit nested belief states representation
• Prompt generation for LLM-based agent control