Search papers, labs, and topics across Lattice.
This paper presents Para-Pipe, a hierarchical mapping framework that optimizes the performance of edge-based deep learning applications on heterogeneous System-on-Chips (SoCs) by integrating intra- and interstage operator parallelism within a pipelined architecture. The framework addresses the latency challenges posed by modern neural networks while maintaining throughput, achieving a significant reduction in interprocessor communication overhead and enhancing energy efficiency. Evaluation results indicate that configurations optimized for throughput under Para-Pipe yield an average energy efficiency improvement of 11.0% over traditional pipelining and 23.3% compared to non-pipelined parallel execution on specific SoC platforms.
Achieving an 11% boost in energy efficiency for edge AI applications by smartly balancing throughput and latency through hierarchical operator parallelism.
As edge-based deep learning applications become more complex, optimizing performance on heterogeneous System-on-Chips (SoCs) presents unique challenges. Traditional pipelining techniques distributing the computation across different on-chip processing units, while effective for throughput, do not address the latency demands posed by modern neural networks with complex interdependencies and extensive operator parallelism. There is a potential in leveraging operator parallelism to enable concurrent execution across multiple processing units, thereby reducing inference latency. However, prioritizing pipelining or parallel execution often necessitates a compromise, where optimizing one performance metric adversely impacts the other. This article introduces Para-Pipe, a hierarchical mapping framework that integrates intra- and interstage operator parallelism within a pipelined architecture. Para-Pipe navigates the tradeoff between throughput and latency by selectively fine-tuning parallelism levels within and across pipeline stages. This strategy can significantly reduce interprocessor communication overhead, significantly improving energy efficiency. Our evaluation demonstrates that Para-Pipe generates multiple Pareto-optimal configurations, achieving a balance between throughput and latency on an Amlogic SoC equipped with ARM big.LITTLE CPUs and GPU, as well as the Black Sesame Technology SoC featuring a deep learning accelerator and two DSPs. More importantly, throughput-optimized configurations under Para-Pipe on Amlogic SoC show an average energy efficiency improvement of 11.0% over purely pipelined strategies and 23.3% relative to nonpipelined parallel execution.
NUS