Search papers, labs, and topics across Lattice.
This paper introduces FuseFSS, a compiler that streamlines secure inference for large language models (LLMs) by replacing the traditional bespoke protocol design with a unified compilation pipeline for fixed-point operators. By efficiently handling nonlinearities and helper operations, FuseFSS achieves significant improvements in speed and communication efficiency, yielding a 1.24x to 1.50x speedup while reducing online communication by 9% to 16%. The method maintains accuracy across BERT and GPT-style models, while also decreasing key-generation time and key size, thereby enhancing the practicality of secure LLM deployment.
Achieving up to 50% faster secure inference for LLMs without sacrificing accuracy could revolutionize the deployment of privacy-preserving AI systems.
Two-server secure inference allows a client to query a hosted large language model (LLM) without revealing prompts or embeddings. Recent GPU systems based on function secret sharing (FSS) make linear layers efficient, but fixed-point nonlinearities and helper operations remain a bottleneck because each operator is typically implemented as a bespoke protocol with its own comparisons, wrap-around corrections, and preprocessing material. We present FuseFSS, a compiler that replaces per-operator protocol design with a single compilation pipeline. For each scalar fixed-point operator, a compact specification lists its interval partition, low-degree arithmetic pieces, and required predicate bits. The compiler emits two batched FSS evaluations on the public masked value: one packed comparison that returns all predicate bits, and one vector interval lookup that returns the active coefficients and constants. Compared to the current state-of-the-art FSS-based GPU secure inference, FuseFSS preserves accuracy while achieving a $1.24\times$--$1.50\times$ end-to-end speedup and reducing online communication by $9\%$--$16\%$ on BERT and GPT-style models; preprocessing is also lighter, with $14\%$--$23\%$ lower key-generation time and $20\%$--$24\%$ smaller keys.
