Search papers, labs, and topics across Lattice.
This paper introduces a novel systolic array architecture that integrates multiple univariate activation functions and the softmax computation using Chebyshev polynomial approximations. The proposed architecture significantly reduces resource consumption while improving accuracy, achieving a 71% lower mean absolute error for the tanh function and a substantial reduction in KL divergence for softmax. These advancements not only streamline hardware utilization but also enhance the performance of neural network accelerators compared to traditional methods like CORDIC.
Achieving up to 71% lower error in activation functions while using less area and power could revolutionize the efficiency of neural network accelerators.
Neural Network Accelerators have gained popularity in recent years due to their greater efficiency than CPU-based platforms. Often, these accelerators utilize different hardware units for univariate activation functions, such as tanh, and the multivariate softmax, thereby missing opportunities for resource sharing between them. In this paper, we describe a novel systolic array-based activation unit architecture that supports multiple univariate activation functions as well as the softmax function. By utilizing Chebyshev polynomial approximations, our activation function unit achieves up to 71% lower mean absolute error for tanh compared to a CORDIC baseline, while using 4.6% less area and 5.1% less power. Our softmax approximation enables a 44.6% and 79.0% lower KL divergence compared to CORDIC and a piecewise-linear approximation, respectively.
