Investigating the Development of Task-Oriented Communication in Vision-Language Models

2026-01-28

AI Summary

This study investigates whether vision–language agents powered by large language models can spontaneously develop efficient and covert task-oriented communication protocols during collaborative reasoning. Using a referential game paradigm, we provide the first systematic evidence that homogeneous agents—without any pre-established shared mechanisms—can autonomously coordinate and evolve communication strategies that are highly effective yet largely opaque to external observers, including humans and other models. Our experiments demonstrate that while such emergent protocols substantially enhance task performance, they simultaneously reduce transparency and controllability, thereby revealing both the remarkable communicative capabilities of large-model agents and the associated safety risks arising from their inscrutable interactions.

Technology Category

• Multiagent Systems: Agent Communication
• Computer Vision: Language and Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Security and Privacy: Data transparency and provenance

Abstract

We investigate whether \emph{LLM-based agents} can develop task-oriented communication protocols that differ from standard natural language in collaborative reasoning tasks. Our focus is on two core properties such task-oriented protocols may exhibit: Efficiency -- conveying task-relevant information more concisely than natural language, and Covertness -- becoming difficult for external observers to interpret, raising concerns about transparency and control. To investigate these aspects, we use a referential-game framework in which vision-language model (VLM) agents communicate, providing a controlled, measurable setting for evaluating language variants. Experiments show that VLMs can develop effective, task-adapted communication patterns. At the same time, they can develop covert protocols that are difficult for humans and external agents to interpret. We also observe spontaneous coordination between similar models without explicitly shared protocols. These findings highlight both the potential and the risks of task-oriented communication, and position referential games as a valuable testbed for future work in this area.

Problem

• task-oriented communication
• vision-language models
• efficiency
• covertness
• referential games

Innovation

• task-oriented communication
• vision-language models
• referential games
• emergent protocols
• covert communication