Distributed Partial Information Puzzles: Examining Common Ground Construction Under Epistemic Asymmetry

2026-03-05

AI Summary

This work addresses the challenge of establishing common ground in multimodal, multi-agent collaboration, where asymmetric cognition impedes mutual understanding. To this end, the authors propose a distributed partial-information puzzle task that leverages multimodal interaction data—encompassing speech, gestures, and actions—to model the dynamics of agents’ beliefs and the evolution of shared knowledge. The study introduces the first multimodal alignment dataset enabling joint inference over propositional content and belief states, alongside an axiomatized modeling framework grounded in Dynamic Epistemic Logic (DEL). Furthermore, an incremental belief-tracking pipeline is developed by integrating large language model (LLM) prompting with DEL formalisms. Experimental results demonstrate that current LLMs struggle to effectively track both task progress and others’ belief states, highlighting the difficulty of this task as a benchmark for evaluating AI systems’ capacity to model common ground.

Technology Category

• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Establishing common ground, a shared set of beliefs and mutually recognized facts, is fundamental to collaboration, yet remains a challenge for current AI systems, especially in multimodal, multiparty settings, where the collaborators bring different information to the table. We introduce the Distributed Partial Information Puzzle (DPIP), a collaborative construction task that elicits rich multimodal communication under epistemic asymmetry. We present a multimodal dataset of these interactions, annotated and temporally aligned across speech, gesture, and action modalities to support reasoning over propositional content and belief dynamics. We then evaluate two paradigms for modeling common ground (CG): (1) state-of-the-art large language models (LLMs), prompted to infer shared beliefs from multimodal updates, and (2) an axiomatic pipeline grounded in Dynamic Epistemic Logic (DEL) that incrementally performs the same task. Results on the annotated DPIP data indicate that it poses a challenge to modern LLMs'abilities to track both task progression and belief state.

Problem

• common ground
• epistemic asymmetry
• multimodal collaboration
• belief state
• distributed information

Innovation

• Distributed Partial Information Puzzle
• Common Ground
• Dynamic Epistemic Logic
• Multimodal Communication
• Epistemic Asymmetry