Search papers, labs, and topics across Lattice.
2
0
2
Pushing KV cache quantization down to 2 bits without sacrificing downstream accuracy is now viable by rotating key and value matrices according to their joint second-order interaction statistics.
NVIDIA's hand-crafted GEMM heuristics leave massive performance on the table: grounding learned kernel selectors in static hardware-behavior estimates cuts selection regret by up to 64.2% without running an autotuning benchmark.
