Xi Peng

Latent communication between LLMs was largely an illusion of interface shortcutting until Draft-KV, which routes genuine drafting KV caches to let a frozen 0.5B model leap from 37% to 78% MMLU-Redux accuracy by tapping an 8B model's latent thoughts.

Pre-trained Softmax ViTs can directly bootstrap linear-attention models without expensive pre-training from scratch: simply copy the operator-agnostic MLP weights and distill the attention routing behavior.

Neural networks exhibit an internal "feeling of error"—manifesting as sharply suppressed concept activations inside Sparse Autoencoders—that reliably flags segmentation failures where standard output uncertainty metrics fail.

ViT weights harbor fully legible, input-invariant concept graphs of the world, unlocking targeted circuit interventions that improve shortcut removal by 11.0% over prior baselines.