Learning to Communicate: Toward End-to-End Optimization of Multi-Agent Language Systems

2026-04-23

AI Summary

This work addresses the limitation of existing large language model–based multi-agent systems, which typically treat communication as a fixed interface and thus struggle to jointly optimize communication and reasoning. The authors propose DiffMAS, a novel framework that enables end-to-end joint optimization of latent communication and collaborative reasoning for the first time. DiffMAS employs parameter-efficient fine-tuning to learn encoding and decoding strategies over agents’ hidden-state trajectories and introduces a learnable communication mechanism grounded in key-value caching. Experimental results demonstrate that DiffMAS significantly outperforms single-agent baselines, text-based communication approaches, and existing latent communication methods across multiple reasoning benchmarks—including AIME24 (26.7% improvement) and GPQA-Diamond (20.2% improvement)—achieving higher reasoning accuracy and enhanced decoding stability.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Multi-agent systems built on large language models have shown strong performance on complex reasoning tasks, yet most work focuses on agent roles and orchestration while treating inter-agent communication as a fixed interface. Latent communication through internal representations such as key-value caches offers a promising alternative to text-based protocols, but existing approaches do not jointly optimize communication with multi-agent reasoning. Therefore we propose DiffMAS, a training framework that treats latent communication as a learnable component of multi-agent systems. DiffMAS performs parameter-efficient supervised training over multi-agent latent trajectories, enabling agents to jointly learn how information should be encoded and interpreted across interactions. Experiments on mathematical reasoning, scientific QA, code generation, and commonsense benchmarks show that DiffMAS consistently improves reasoning accuracy and decoding stability over single-agent inference, text-based multi-agent systems, and prior latent communication methods, achieving 26.7% on AIME24, 20.2% on GPQA-Diamond, and consistent gains across reasoning benchmarks.

Problem

• multi-agent systems
• latent communication
• end-to-end optimization
• language models
• inter-agent communication

Innovation

• latent communication
• multi-agent systems
• end-to-end optimization
• parameter-efficient training
• large language models