Search papers, labs, and topics across Lattice.
This paper introduces StateBridge, a training-free method for aligning hidden states in large language model (LLM) multi-agent systems, enabling direct latent communication without the need for discrete tokenization. By employing a closed-form orthogonal transformation to align the sender's final-layer hidden states with the receiver's input space, StateBridge effectively preserves information that would otherwise be lost in text conversion. The method demonstrates superior performance across various tasks, achieving the best or tied-best results on 22 out of 26 model-task pairs compared to existing approaches.
StateBridge reveals that training-free hidden-state alignment can significantly enhance communication efficiency in LLM multi-agent systems, outperforming traditional methods.
Large language model based multi-agent systems usually communicate in text, i.e., using discrete tokens. However, text introduces a discrete bottleneck. Converting the sender's continuous hidden states into discrete tokens discards information that token identities alone cannot capture. Recent work proposes latent communication as an alternative, where agents transmit hidden representations directly without converting them to text. However, existing latent methods either inject working memory layer by layer across the transformers, or require trained projectors that limit portability. We propose StateBridge, a training-free latent communication approach that aligns the sender's final-layer hidden states to the receiver's input space via a closed-form orthogonal transformation. Lightweight norm calibration and vocabulary anchoring ensure compatibility with the pretrained input distribution. The aligned states are prepended to the input of the receiver agent as a continuous prefix. We evaluate StateBridge on math reasoning, code generation, and question answering with four models from two families. StateBridge achieves the best or tied-best score on 22 out of 26 model-task pairs, consistently outperforming the strongest baseline.
