Search papers, labs, and topics across Lattice.
4
0
6
27
Reward modeling inference costs plummet by up to 7x by performing judgments entirely in a compressed latent space, bypassing token-by-token generation.
Forget content, remember position: crafting pseudo-queries based on token position alone yields surprisingly effective KV cache compression for LLMs, rivaling methods that analyze input semantics.
Achieve 11.8x faster reasoning with 80% KV cache compression by estimating token importance directly from FlashAttention's intermediate results – no extra compute needed.
