Successful Misunderstandings: Learning to Coordinate Without Being Understood

2025-09-29

AI Summary

This paper investigates how heterogeneous agents (e.g., humans and AI) self-organize and evolve communication protocols under conditions of no shared environmental observables—relying solely on signals and interactive feedback. Method: Building on the signaling game framework, we integrate reinforcement learning with dynamic policy updating to simulate multi-agent co-evolutionary dynamics. Contribution/Results: We identify a phenomenon termed “successful misinterpretation”: agents achieve high coordination efficiency despite inconsistent signal interpretations; however, such inconsistencies remain latent and irreparable, impeding integration of new agents. Furthermore, we formally establish that a fully connected triadic topology is the minimal structural condition for the emergence of stable, scalable shared semantics. These findings demonstrate that coordination does not entail mutual understanding, thereby providing theoretical foundations and architectural constraints for trustworthy human–AI communication in low-consensus environments.

Technology Category

• Multiagent Systems: Agent Communication
• Game Theory and Economic Paradigms: Coordination and Collaboration
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The main approach to evaluating communication is by assessing how well it facilitates coordination. If two or more individuals can coordinate through communication, it is generally assumed that they understand one another. We investigate this assumption in a signaling game where individuals develop a new vocabulary of signals to coordinate successfully. In our game, the individuals do not have common observations besides the communication signal and outcome of the interaction, i.e. received reward. This setting is used as a proxy to study communication emergence in populations of agents that perceive their environment very differently, e.g. hybrid populations that include humans and artificial agents. Agents develop signals, use them, and refine interpretations while not observing how other agents are using them. While populations always converge to optimal levels of coordination, in some cases, interacting agents interpret and use signals differently, converging to what we call successful misunderstandings. However, agents of population that coordinate using misaligned interpretations, are unable to establish successful coordination with new interaction partners. Not leading to coordination failure immediately, successful misunderstandings are difficult to spot and repair. Having at least three agents that all interact with each other are the two minimum conditions to ensure the emergence of shared interpretations. Under these conditions, the agent population exhibits this emergent property of compensating for the lack of shared observations of signal use, ensuring the emergence of shared interpretations.

Problem

• Investigating whether successful coordination requires mutual understanding in communication
• Studying how agents develop signals without shared observations or aligned interpretations
• Identifying conditions for emergent shared interpretations in multi-agent communication systems

Innovation

• Agents develop new vocabulary without shared observations
• Successful coordination achieved through misaligned signal interpretations
• Minimum three interacting agents ensure shared interpretations emerge