Search papers, labs, and topics across Lattice.
This paper introduces Factorized Encrypted Scan-Contract (FESC), a hybrid FHE-MPC system designed for efficient private long-context inference using selective state-space models (SSMs). By optimizing the handling of encrypted transitions and state residency, FESC achieves linear-time inference while maintaining compatibility across various SSM architectures. The system demonstrates the ability to execute long-document inference natively on a single GPU for sequences up to 2,048 tokens, achieving near-plaintext accuracy with a manageable memory footprint.
FESC achieves private long-document inference on a single GPU for sequences up to 2,048 tokens, setting a new standard for efficiency and accuracy in encrypted machine learning.
Processing long, sensitive documents with machine-learning models requires efficient, privacy-preserving long-context inference. Prior private inference systems optimize or distribute encrypted Transformer attention, but its quadratic token-pair work remains the bottleneck as sequence length grows. Selective state-space models (SSMs) offer linear-time recurrence, yet direct encrypted implementation incurs linear multiplicative depth, sequence-wide state residency, or dense FHE-MPC conversion. We present Factorized Encrypted Scan-Contract (FESC), a hybrid FHE-MPC system for private long-context selective SSM inference. Its factorized scan-contract keeps input-dependent transitions compact across conversion boundaries, composes them without dense expansion, streams state chunks on demand, and contracts outputs before conversion. We demonstrate interface compatibility of the scan-contract implementation across invariant and selective SSM architectures. For our Mamba-2 instantiation, we design GPU-optimized CKKS kernels for linear computations, MPC protocols for SiLU, softplus, exponential, and RMSNorm, with approximation-aware fine-tuning. To our knowledge, FESC is the first private long-document inference system to complete native end-to-end execution at $L \geq 1{,}024$ on a single GPU. At $L = 2{,}048$, a 12-layer Mamba-base model completes inference in 77.3 minutes on one A100 GPU with a peak memory footprint of 32.7 GB, while maintaining near-plaintext accuracy on the evaluated long-document tasks.
