Search papers, labs, and topics across Lattice.
4
0
6
4
INT4 KV caching for MLA models is now possible, achieving 3.59x compression at 128K context and enabling 5.168x higher throughput in cache-pressure serving workloads.
Pre-attention spikes and inter-spike plateaus reveal a surprising organization in hybrid linear attention models that could redefine our understanding of activation dynamics in LLMs.
Achieving over 99% performance retention while significantly accelerating large recommendation models by intelligently merging experts based on functional similarity.
