Heng Wang

Complex scoring algorithms for KV cache compression are largely unnecessary: protecting the initial prompt and dropping reasoning tokens completely at random matches state-of-the-art accuracy with up to 43% higher serving throughput.

QUMem revolutionizes user-state inference by enabling LLMs to retrieve contextually relevant memories that evolve over time, outperforming existing methods on key benchmarks.

Query-conditioned reuse boosts agent success by 10.7 points while slashing token usage by nearly 50%, transforming how we leverage past experiences in AI tasks.