Search papers, labs, and topics across Lattice.
Affiliation:
2
0
4
2-bit KV cache quantization for Looped Transformers achieves 80% memory savings and 4x throughput gains with near-BF16 accuracy by quantizing residuals relative to the final loop's KV states.
