Search papers, labs, and topics across Lattice.
The University of Hong Kong
3
0
6
3
INT4 KV caching for MLA models is now possible, achieving 3.59x compression at 128K context and enabling 5.168x higher throughput in cache-pressure serving workloads.
Squeeze 3D vision transformers: XStreamVGGT slashes memory consumption by 4.42x and accelerates inference by 5.48x via pruning and quantization of the KV cache, all with negligible performance loss.
