Search papers, labs, and topics across Lattice.
2
1
4
3
Pruning LLMs for decoding speedup is significantly improved by calibrating sparsity with actual generation activations, not just static text, and optimizing SpMV kernels.
LLMs can maintain near-perfect accuracy on long sequences with only 25% of the KV cache, thanks to a novel semantic clustering approach that dramatically improves CPU-GPU offloading.
