Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores the optimization of collaborative inference for large deep neural networks (DNNs) deployed on resource-constrained IoT devices by partitioning the DNN execution between user devices and edge servers. The authors formulate the optimization problem as a Markov decision process (MDP) and employ a multiagent proximal policy optimization (MAPPO) framework to enhance learning stability in complex action spaces. Experimental results demonstrate that their approach significantly improves the latency-energy tradeoff, achieving lower end-to-end latency while maintaining or reducing energy consumption compared to baseline methods.
Collaborative inference optimization can slash end-to-end latency while keeping energy consumption in check for large DNNs on IoT devices.
Deploying large deep neural networks (DNNs) on resource-constrained Internet of Things (IoT) devices is often limited by high inference latency and energy consumption. This article studies collaborative inference between user devices and edge servers (ESs), where a deep neural network (DNN) is partitioned and executed across both sides under shared wireless resources. We jointly optimize the partition point across layers, uplink channel allocation and transmission power control, and formulate the coupled decisions as a Markov decision process (MDP). Under centralized training with decentralized execution (CTDE), we develop a multiagent proximal policy optimization (MAPPO) framework with a multihead critic and profiling-aware state design to stabilize learning under large hybrid action spaces. To validate practicality, we implement a containerized experimental platform that emulates resource-limited IoT devices and GPU-enabled ESs, enabling repeatable measurements of end-to-end latency and energy. Experiments with representative DNNs show that the proposed method consistently improves the latency–energy tradeoff over baseline algorithms, achieving lower end-to-end latency and comparable or lower energy consumption across diverse model structures and computation patterns.