Search papers, labs, and topics across Lattice.
This paper introduces GIFT, a GPU Information Flow Tracking system designed to enforce user data isolation in large language model (LLM) serving frameworks, addressing the critical need for privacy protection in shared infrastructures. By leveraging encryption-as-isolation and static flow analysis, GIFT allows for secure data processing without modifying existing serving logic, achieving minimal overhead. The implementation of GIFT and its extension, GIFT-CC, demonstrates effective isolation with only a 4-10.7% throughput overhead while maintaining latency levels, making it a practical solution for LLM operators.
GIFT achieves user data isolation in LLM serving with minimal overhead, ensuring privacy without compromising performance.
LLM serving frameworks process large volumes of user data--often containing sensitive information--on shared infrastructure. Ensuring isolation between users who share the same serving framework (on CPUs) and LLM operators (on GPUs) is critical for privacy protection. This paper presents GIFT, a GPU Information Flow Tracking system that enforces user data isolation in LLM serving with minimal overhead. Moreover, the design of GIFT is non-intrusive and allows CPU-side serving frameworks to evolve freely. It rests on two key insights. First, encryption-as-isolation leverages the observation that CPU components only orchestrate data flow, not content manipulation; thus, per-user encryption can provide isolation without modifying serving logic. Second, GPU kernels exhibit limited and predictable information flows, enabling static flow analysis. GIFT precomputes information flow rules for each kernel and uses decoupled flow tracking, avoiding instrumentation or GPU stalls. Furthermore, we extend GIFT to GIFT-CC, which integrates confidential computing to protect against untrusted operating systems and hypervisors (LLM service providers). Implemented on vLLM and DistServe, GIFT and GIFT-CC enforce user data isolation with a 4-10.7% throughput overhead while maintaining the same latency level.
