Search papers, labs, and topics across Lattice.
Affiliation:
4
0
8
2-bit KV cache quantization for Looped Transformers achieves 80% memory savings and 4x throughput gains with near-BF16 accuracy by quantizing residuals relative to the final loop's KV states.
LLM agents can now reliably track and fulfill complex deliverable requirements across numerous files without losing context, thanks to an environment-side "tab" that acts as a persistent, shared task ledger.
ProtoKV boosts streaming video understanding accuracy by up to 12.5 points in high-delay scenarios by rethinking how we manage memory.
