Search papers, labs, and topics across Lattice.
This paper addresses the computational overhead associated with key-switching in privacy-preserving two-party Transformer inference, particularly when using Fully Homomorphic Encryption (FHE). By introducing a preprocessing-assisted method that decomposes attention into precomputable operations and online interactions, the authors effectively reduce the key-switching burden during online inference. Their approach, which includes a fused key-switch technique, not only minimizes computational costs but also offers flexible storage-communication trade-offs, enhancing the practicality of secure Transformer models without altering existing packing strategies.
Key-switching overhead in secure Transformer inference can be drastically reduced by offloading computations to a preprocessing phase, enabling efficient deployment even under resource constraints.
In secure two-party Transformer inference, linear layers are typically evaluated using Fully Homomorphic Encryption (FHE) through plaintext-ciphertext or ciphertext-ciphertext matrix multiplications, where key switching primarily occurs and dominates computational overhead in both FHE-based and hybrid FHE-MPC systems. Existing optimizations rely heavily on packing-specific algorithms, limiting their general applicability. Targeting this overhead from a packing-independent perspective, we propose a preprocessing-assisted method for secure attention computation. By decomposing attention into precomputable operations and online interactions, this method reduces online inference-phase key switching without modifying existing packing strategies. However, the first method shifting key switching offline introduces additional storage requirements. To address this, we propose storage-communication trade-off techniques that replace large precomputed ciphertexts with modest online communication, enabling flexible deployment under varying resource constraints. While ciphertext-ciphertext matrix multiplication is offloaded to the preprocessing phase in hybrid schemes and the first layer of FHE-based schemes, these operations still persist in the offline stage and subsequent FHE layers. To further optimize it, we propose a fused key-switch technique targeting the multiplication-followed-by-rotation pattern, which frequently arises in existing RNS-CKKS matrix multiplication schemes. By combining relinearization and rotation into a single procedure, this technique reduces the associated computation costs. Analytical evaluations demonstrate that our proposed techniques significantly reduce online key-switch overhead and provide flexible trade-offs between storage and communication without requiring modifications to existing packing strategies.
