HiVAE: Hierarchical Latent Variables for Scalable Theory of Mind

2026-02-18

AI Summary

Existing theory of mind (ToM) approaches struggle to scale to complex real-world scenarios and are often confined to small grid-world environments. This work proposes HiVAE, a hierarchical variational autoencoder inspired by the human belief-desire-intention cognitive architecture, which introduces hierarchical latent variables into scalable ToM modeling for the first time. By employing a self-supervised alignment strategy to enhance the semantic interpretability of latent representations, HiVAE significantly outperforms baseline models on a large-scale campus navigation task comprising 3,185 nodes. The model not only improves inference of agents’ implicit goals and mental states but also reveals critical challenges and new research directions concerning the explicit alignment between latent space structures and genuine psychological states.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Theory of mind (ToM) enables AI systems to infer agents' hidden goals and mental states, but existing approaches focus mainly on small human understandable gridworld spaces. We introduce HiVAE, a hierarchical variational architecture that scales ToM reasoning to realistic spatiotemporal domains. Inspired by the belief-desire-intention structure of human cognition, our three-level VAE hierarchy achieves substantial performance improvements on a 3,185-node campus navigation task. However, we identify a critical limitation: while our hierarchical structure improves prediction, learned latent representations lack explicit grounding to actual mental states. We propose self-supervised alignment strategies and present this work to solicit community feedback on grounding approaches.

Problem

• Theory of Mind
• scalability
• latent representation
• mental states
• grounding

Innovation

• Hierarchical VAE
• Theory of Mind
• Latent Representation
• Self-supervised Alignment
• Scalable Reasoning