Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To eliminate the dominant latency bottleneck of plaintext-ciphertext multiplications in packed fully homomorphic encryption (FHE), the authors introduce FIONA, an offline optimizer for packing-aware selective weight ternarization. The framework groups weights according to the underlying ciphertext packing layout, enforcing shared ternary values only within error-tolerant groups to replace multiplications with additions while retaining full precision on sensitive paths and lowering activation polynomial degrees. Across VGG11, ViT, and BERT, this hybrid execution cuts plaintext-ciphertext multiplications by 53.4–79.5% and achieves up to 2.38× end-to-end inference speedups with under 1% accuracy loss.
Aligning ternary weight quantization directly with SIMD ciphertext packing layouts eliminates up to 80% of expensive FHE multiplications, unlocking sub-1% accuracy loss speedups across both ViTs and BERT.
Fully homomorphic encryption (FHE) enables neural network inference directly on encrypted inputs, but it remains orders of magnitude slower than plaintext in- ference. Applying the server's plaintext weights to encrypted activations involves plaintext-ciphertext multiplications (PMult) and accounts for more than half of inference time in recent systems. Ternary quantization can replace these multipli- cations with additions and subtractions, but the savings rarely materialize under packed execution. A single PMult applies a weight group fixed by the packing layout and can be avoided only when all its weights share the same ternary value. Ternarizing all groups, however, largely degrades accuracy. We present FIONA, an offline optimizer that selectively ternarizes weights within a given packing layout based on the estimated effect of ternary conversion on the model's performance. FIONA encourages a shared ternary value within each weight group and retains full-precision weights for sensitive groups, so ternar- ized and full-precision paths coexist within a layer. It then compiles these hybrid operators exactly, applying common scaling factors once to accumulated inputs and reusing sums across outputs. Weight ternarization can also narrow the input ranges of downstream polynomials. FIONA fits lower-degree replacements under a cumulative accuracy budget, reducing multiplicative depth and bootstrapping. On VGG11, ViT, and BERT, FIONA reduces PMult operations by 53.4-79.5% and accelerates end-to-end encrypted inference by 2.38x, 1.68x, and 1.84x, re- spectively, with less than 1% accuracy loss across all three models.