Search papers, labs, and topics across Lattice.
This paper introduces Cooperative Autodidactic NeuroSurgeon (CANS), a framework for optimizing multiuser collaborative edge inference by enabling devices to adaptively learn optimal deep neural network (DNN) partitions through shared feedback. The method addresses the challenges posed by fluctuating wireless links and diverse device capabilities, leveraging a novel FedLinUCB-DW algorithm to group similar devices and utilize offline inference experiences for improved online performance. Empirical results show that CANS significantly reduces inference latency, achieving up to a 50% improvement over non-cooperative baselines in real-world prototype experiments.
CANS slashes inference latency by up to 50% in collaborative edge computing by enabling devices to learn optimal DNN partitions through shared feedback.
Recently, mobile edge computing (MEC)-enabled collaborative deep neural network (DNN) inference has emerged as a promising approach for delivering intelligent services to resource-constrained mobile devices. A representative scenario is multi-user collaborative edge inference, where distinct devices independently partition their DNN models and offload backend computation to a common edge server over wireless networks. However, determining the optimal DNN partition for each device is challenging due to unknown and time-varying system conditions, including fluctuating wireless links and diverse device capabilities. To address this problem, we propose Cooperative Autodidactic NeuroSurgeon (CANS), a collaborative edge inference framework that enables devices to adaptively learn optimal DNN partitions by sharing informative feedback during online inference. To handle the challenge of device heterogeneity and better leverage offline inference experience, we integrate a novel FedLinUCB-DW algorithm that groups devices of the same type and warm-starts online exploration using local offline early-exit inference experience. Furthermore, we provide theoretical guarantees for FedLinUCB-DW by deriving the regret upper bound. We also validate our method on both a simulated environment and a hardware prototype system. Empirical evaluations demonstrate that CANS achieves lower inference latency compared to state-of-the-art baselines. Especially, in prototype experiments on two edge devices, the proposed CANS reduced average inference latency by up to 50% compared to the non-cooperative baseline.
