Search papers, labs, and topics across Lattice.
This paper introduces a framework that optimally utilizes underused AI computation resources at the edge by converting traditional computing tasks into neural network models through a representative neural architecture search (NAS) method. By implementing a runtime scheduler, the framework offloads these approximate tasks to AI chips during idle periods, enhancing performance without disrupting primary workloads. Experimental results demonstrate significant performance gains in edge processing tasks, addressing the inefficiencies of both specialized AI chips and general-purpose processors.
Unused AI computation at the edge can be harnessed to boost performance in traditional tasks without sacrificing the efficiency of primary workloads.
With the widespread adoption of AI in various IoT scenarios such as smart sensing and processing, AI chips have become a common component at the edge. These chips are typically specialized for structured neural network (NN) processing and are designed to meet peak workload demands. However, they are often underutilized and suffer from considerable computational waste due to temporal or spatial redundancy in processing. Conversely, general-purpose processing engines at the edge may struggle with compute-intensive tasks such as signal processing and complex numerical operations because of stringent resource constraints. To address this imbalance, we propose a framework that harvests unused AI computation resources using general-purpose approximation techniques. The core idea is to automatically convert traditional computing tasks into neural network models via a representative neural architecture search (NAS) method. These approximate versions of general-purpose tasks are then deployed on AI engines during their idle periods. Specifically, we introduce a runtime scheduler that offloads these tasks to AI chips without compromising the performance of primary AI workloads, thereby alleviating the burden on general-purpose processors. Experiments on a representative AIoT processor show that our proposed AI computation harvesting strategy delivers substantial performance improvements across a set of edge processing tasks.
