Chayne Thrash

LLM inference cost plummets as a novel output-aware KV cache retrieval method bypasses attention score approximation errors, achieving competitive performance with minimal overhead.

Shrinking an LLM's residual stream based on activation variance alone discards the wrong directions—weighting pruned subspaces by downstream output sensitivity dramatically improves compressed model performance without sacrificing closed-form computational efficiency.