Search papers, labs, and topics across Lattice.
. University of Southern California , USA
5
0
9
9
HeteroFold unlocks $10.7\times$ faster cross-family KV cache transfer for LLMs, eliminating receiver prefill and matching text-based communication performance.
This work proposes ASPIRE, a non-synchronized batched self-speculative decoding framework built on three components, which achieves speedup in decoding throughput over autoregressive baselines and improves average speedup by approximately $27\% over the strongest prior self-speculative baselines.
OTRO reduces tokenizer latency to just 4.5% overhead while effectively safeguarding against side-channel attacks in LLMs.
A novel differentially private RAG algorithm, DP-KSA, achieves a strong privacy-utility tradeoff by extracting and augmenting prompts with differentially private keywords derived from an ensemble of LLM responses.
Decomposing linear layers in MPC inference can actually speed things up, but only if you carefully manage the extra communication rounds and truncation steps.
