Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors reveal that prior latent LM-to-LM communication methods achieve spurious gains that persist even when swapping in messages from unrelated tasks, prompting them to design Draft-KV to enforce genuine inter-model communication. Draft-KV linearly projects the key-value states generated during a frozen sharer's answer drafting phase into a side memory accessed by the frozen receiver through a gated attention branch, optimized via progressive reconstruction and mismatch-penalized supervision. With only 1.05M interface parameters (348x fewer than C2C), this mechanism elevates a frozen Qwen2.5-0.5B receiver from 37.45% to 78.04% on MMLU-Redux via a Qwen3-8B sharer, with performance collapsing to 36.40% under mismatched inputs to confirm true content utilization.
Latent communication between LLMs was largely an illusion of interface shortcutting until Draft-KV, which routes genuine drafting KV caches to let a frozen 0.5B model leap from 37% to 78% MMLU-Redux accuracy by tapping an 8B model's latent thoughts.
Latent communication passes internal states between language models instead of decoded text, but higher receiver accuracy does not show that the receiver used the message content. Across five method-dataset pairs, replacing each message with one from an unrelated question changes accuracy by at most 0.60 points, even when communication adds 15.44 points over the receiver alone. Thus the interface can supply the gain while making the sharer dispensable. Draft-KV instead sends the key-value states formed while the sharer drafts an answer to the current question. Linear projections place these states in a side memory read through a gated attention branch, and progressive training moves from message reconstruction to answer supervision under a guard on harm from mismatched messages. Both models remain frozen and the interface trains 1.05M parameters, 348x fewer than C2C. With a Qwen3-8B sharer, a frozen Qwen2.5-0.5B-Instruct receiver reaches 78.04% on MMLU-Redux, versus 37.45% alone and 36.40% with reassigned messages. At fixed interface size, scaling the sharer from 0.6B to 8B raises accuracy from 46.11% to 78.04%; communication also transfers to held-out tasks and can exceed both models when each holds different evidence.