Search papers, labs, and topics across Lattice.
Affiliation:
2
0
2
Heuristic KV cache pruning routinely fails under heavy compression because it ignores softmax curvature, but treating attention as a nonlinear Gaussian channel reveals exactly which tokens preserve information capacity.
Adjusting gate sharpness based on routing confidence leads to significant performance gains in Mixture-of-Experts models without the burden of extra parameters.
