Search papers, labs, and topics across Lattice.
The paper introduces CacheSolidarity, a system designed to mitigate Automatic Prefix Caching (APC) side channels in multi-tenant LLM serving systems. CacheSolidarity monitors cache reuse across users, identifies suspicious sharing patterns, and selectively isolates prefixes to prevent information leakage. Evaluations demonstrate that CacheSolidarity achieves up to 70% higher cache reuse and 30% lower inference latency compared to defenses that disable APC entirely.
Secure multi-tenant LLM serving without sacrificing performance is now possible: CacheSolidarity selectively isolates prefixes, boosting cache reuse by up to 70% and cutting inference latency by 30% compared to blunt-force defenses.
Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference. APC works by reusing previously computed states for the beginning part of a request (prefix), when another request starts with the same text. While APC improves throughput, it introduces timing side channels: cache hits are faster than misses, creating observable latency differences. In multi-tenant systems, attackers can exploit these differences to infer sensitive information, e.g., by incrementally reconstructing another user's request by observing hit/miss patterns. Current defenses take a sledgehammer approach: they disable APC and cache sharing, isolating users, and sacrificing efficiency for regular users. This paper presents CacheSolidarity, a system that secures multi-tenant LLM serving systems against APC side channels without sacrificing performance and efficiency. CacheSolidarity monitors cache reuse across users, flags suspicious sharing, and selectively isolates prefixes, restricting their reuse only when necessary. Evaluation shows that CacheSolidarity enables up to 70% higher cache reuse and 30% lower inference latency compared to existing defenses that isolate users. CacheSolidarity's lightweight design demonstrates how security in LLM serving does not have to come at the cost of unnecessarily reduced performance or unbearable overheads.
