Search papers, labs, and topics across Lattice.
Hong Kong University of Science and Technology
2
0
4
LLMs can now compress their KV cache more effectively by dynamically synthesizing soft tokens tailored to the input, preserving crucial context that's otherwise lost with static methods.
Multi-chiplet architectures can unlock significant speedups and memory savings for low-batch MoE inference by dynamically scheduling expert computations across high-bandwidth die-to-die links.
