Search papers, labs, and topics across Lattice.
This paper introduces a latency-optimal adaptive split inference framework designed for privacy-preserving collaboration between cloud, edge, and IoT end devices, which are constrained by computational resources. By utilizing fully homomorphic encryption (FHE) and strategically splitting model execution between the end device and cloud/edge, the framework significantly reduces inference latency while maintaining accuracy. Experimental results demonstrate that the proposed method achieves up to 12.9 times speedup compared to full-cloud FHE and 3.9 times over block-level alternatives, with practical latencies of approximately 1033.279 s/sample on CIFAR-10 and 1023.429 s/sample on PathMNIST.
Achieving nearly 13 times faster inference while ensuring privacy could redefine how IoT devices handle sensitive data in real-time applications.
Internet of Things (IoT) end devices are increasingly expected to support privacy-sensitive batch inference, yet their limited computational resources often make full local execution of convolutional neural networks impractical. This paper presents a latency-optimal adaptive split inference framework for privacy-preserving cloud-edge-end collaboration. The end device acts as the trust anchor, executes the plaintext model prefix, encrypts the split activation using fully homomorphic encryption (FHE), and keeps the secret key locally, while the edge and cloud execute assigned model segments only on FHE ciphertexts. We formulate collaborative encrypted inference as a split-pair selection problem over an end-side split point and an edge-side termination point. The proposed planner jointly models plaintext prefix execution, encryption, communication, edge-side FHE execution, and cloud-side FHE completion, and supports both convolution-level and block-level split granularities. Experiments on CIFAR-10 and PathMNIST show that the proposed convolution-level collaborative scheme achieves amortized end-to-end speedups of approximately 12.9 times over full-cloud FHE and 3.9 times over the block-level alternative, while preserving the corresponding plaintext-model accuracy. Including modeled communication, the amortized latencies are 1033.279 s/sample on CIFAR-10 and 1023.429 s/sample on PathMNIST.
