Enabling Agents to Communicate Entirely in Latent Space

2025-11-12

AI Summary

Natural language as a communication medium among LLM agents suffers from fundamental limitations: discrete tokenization severely compresses and distorts rich latent state information, hindering collaborative reasoning. To address this, we propose Interlat—the first full-latent-space agent communication paradigm leveraging the final-layer hidden states of LLMs, eliminating natural language generation entirely and enabling direct information compression, transmission, and joint reasoning in continuous latent space. Its core innovation is the first end-to-end, tokenization-free direct latent-state interconnection between agents. Experiments demonstrate that Interlat significantly outperforms fine-tuned chain-of-thought prompting and single-agent baselines, accelerating inference while maintaining or even improving task performance. Moreover, it preserves critical semantic content and reasoning pathways more completely than language-based alternatives.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Agent Communication
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

While natural language is the de facto communication medium for LLM-based agents, it presents a fundamental constraint. The process of downsampling rich, internal latent states into discrete tokens inherently limits the depth and nuance of information that can be transmitted, thereby hindering collaborative problem-solving. Inspired by human mind-reading, we propose Interlat (Inter-agent Latent Space Communication), a paradigm that leverages the last hidden states of an LLM as a representation of its mind for direct transmission (termed latent communication). An additional compression process further compresses latent communication via entirely latent space reasoning. Experiments demonstrate that Interlat outperforms both fine-tuned chain-of-thought (CoT) prompting and single-agent baselines, promoting more exploratory behavior and enabling genuine utilization of latent information. Further compression not only substantially accelerates inference but also maintains competitive performance through an efficient information-preserving mechanism. We position this work as a feasibility study of entirely latent space inter-agent communication, and our results highlight its potential, offering valuable insights for future research.

Problem

• Overcoming information loss from latent state discretization
• Enhancing collaborative problem-solving through direct latent transmission
• Accelerating inference while preserving compressed latent information

Innovation

• Direct latent space communication between agents
• Compression via latent space reasoning for efficiency
• Utilizing hidden states as mind representations