Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces DEL, a framework for differentially private and communication-efficient split inference of large language models (LLMs). DEL uses an embedding projection module and differentially private stochastic quantization to reduce communication overhead while preserving privacy. It then employs soft prompts on the server side to mitigate utility degradation caused by the privacy mechanisms, eliminating the need for local models.
Soft prompts can dramatically improve the privacy-utility tradeoff in split LLM inference, enabling practical private LLM usage without local models.
Large Language Models (LLMs) have achieved remarkable performance and received significant research interest. The enormous computational demands, however, hinder the local deployment on devices with limited resources. The current prevalent LLM inference paradigms require users to send queries to the service providers for processing, which raises critical privacy concerns. Existing approaches propose to allow the users to obfuscate the token embeddings before transmission and utilize local models for denoising. Nonetheless, transmitting the token embeddings and deploying local models may result in excessive communication and computation overhead, preventing practical implementation. In this work, we propose \textbf{DEL}, a framework for \textbf{D}ifferentially private and communication \textbf{E}fficient \textbf{L}LM split inference. More specifically, an embedding projection module and a differentially private stochastic quantization mechanism are proposed to reduce the communication overhead in a privacy-preserving manner. To eliminate the need for local models, we adapt soft prompt at the server side to compensate for the utility degradation caused by privacy. To the best of our knowledge, this is the first work that utilizes soft prompt to improve the trade-off between privacy and utility in LLM inference, and extensive experiments on text generation and natural language understanding benchmarks demonstrate the effectiveness of the proposed method.