Search papers, labs, and topics across Lattice.
This paper introduces GIFT, a GPU Information Flow Tracking system designed to enforce user data isolation in large language model (LLM) serving frameworks, which is crucial for protecting sensitive user information. By leveraging encryption-as-isolation and static flow analysis of GPU kernels, GIFT achieves minimal overhead while allowing CPU-side frameworks to evolve independently. The implementation shows that GIFT and its extension, GIFT-CC, maintain throughput with only a 4-10.7% overhead, effectively safeguarding user data against potential privacy breaches.
GIFT enforces user data isolation in LLM serving with less than 11% throughput overhead, revolutionizing privacy protection in shared infrastructures.
LLM serving frameworks process large volumes of user data--often containing sensitive information--on shared infrastructure. Ensuring isolation between users who share the same serving framework (on CPUs) and LLM operators (on GPUs) is critical for privacy protection. This paper presents GIFT, a GPU Information Flow Tracking system that enforces user data isolation in LLM serving with minimal overhead. Moreover, the design of GIFT is non-intrusive and allows CPU-side serving frameworks to evolve freely. It rests on two key insights. First, encryption-as-isolation leverages the observation that CPU components only orchestrate data flow, not content manipulation; thus, per-user encryption can provide isolation without modifying serving logic. Second, GPU kernels exhibit limited and predictable information flows, enabling static flow analysis. GIFT precomputes information flow rules for each kernel and uses decoupled flow tracking, avoiding instrumentation or GPU stalls. Furthermore, we extend GIFT to GIFT-CC, which integrates confidential computing to protect against untrusted operating systems and hypervisors (LLM service providers). Implemented on vLLM and DistServe, GIFT and GIFT-CC enforce user data isolation with a 4-10.7% throughput overhead while maintaining the same latency level.
