StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems

2026-08-13

AI Summary

This work addresses the severe loss of hidden-state information in large language model (LLM)-based multi-agent systems caused by discrete, token-level communication. To mitigate this issue, the authors propose a training-free hidden-state alignment mechanism that leverages a closed-form orthogonal transformation to map the sender’s final-layer hidden states into the receiver’s input space, injecting them as continuous prefixes. This approach requires no layer-wise intervention or dedicated projection modules, offering a lightweight and transferable solution. Combined with norm calibration and lexical anchoring strategies, the method achieves state-of-the-art or tied-best performance in 22 out of 26 model-task combinations across mathematical reasoning, code generation, and question answering benchmarks, significantly outperforming current strongest baselines.

Abstract

Large language model based multi-agent systems usually communicate in text, i.e., using discrete tokens. However, text introduces a discrete bottleneck. Converting the sender's continuous hidden states into discrete tokens discards information that token identities alone cannot capture. Recent work proposes latent communication as an alternative, where agents transmit hidden representations directly without converting them to text. However, existing latent methods either inject working memory layer by layer across the transformers, or require trained projectors that limit portability. We propose StateBridge, a training-free latent communication approach that aligns the sender's final-layer hidden states to the receiver's input space via a closed-form orthogonal transformation. Lightweight norm calibration and vocabulary anchoring ensure compatibility with the pretrained input distribution. The aligned states are prepended to the input of the receiver agent as a continuous prefix. We evaluate StateBridge on math reasoning, code generation, and question answering with four models from two families. StateBridge achieves the best or tied-best score on 22 out of 26 model-task pairs, consistently outperforming the strongest baseline.

Problem

• latent communication
• multi-agent systems
• hidden-state alignment
• discrete bottleneck
• LLM

Innovation

• latent communication
• training-free alignment
• hidden-state transfer
• orthogonal transformation
• multi-agent LLM systems