Search papers, labs, and topics across Lattice.
Affiliation:
1
0
2
Reclaiming memory from idle KV cache during decode phases yields negligible latency improvements, challenging assumptions about prefill chunk sizes in LLM serving.
