Search papers, labs, and topics across Lattice.
Affiliation:
2
0
4
2-bit KV cache quantization for Looped Transformers achieves 80% memory savings and 4x throughput gains with near-BF16 accuracy by quantizing residuals relative to the final loop's KV states.
Hallucinations aren't always due to a lack of knowledge; larger LLMs often "know" the right answer but fail to commit to it, scattering probability across multiple surface forms.
