Search papers, labs, and topics across Lattice.
This paper introduces the Minimalist Integer Vector Engine (MIVE), a programmable architecture designed to accelerate the computation of LayerNorm, RMSNorm, and Softmax operations in Large Language Models (LLMs). By leveraging shared computational patterns among these non-linear vector normalization functions, MIVE significantly reduces resource duplication and enhances silicon utilization compared to traditional dedicated hardware implementations. Experimental results demonstrate that MIVE achieves superior area and hardware efficiency, making it a promising solution for meeting the stringent latency and power demands of modern AI applications.
MIVE achieves unprecedented hardware efficiency by unifying the execution of critical normalization operations, slashing resource overhead in LLM inference.
The rapid growth of Large Language Models (LLMs) has intensified the need for specialized hardware accelerators that can satisfy stringent inference latency and power constraints. Although matrix multiplications dominate the overall computational workload, non-linear vector normalization operations, such as LayerNorm, RMSNorm and Softmax can become critical hardware bottlenecks. Existing accelerators typically implement these functions using dedicated hardware blocks, leading to duplicated resources and inefficient silicon utilization. To address this limitation, we propose a Minimalist Integer Vector Engine (MIVE), a programmable architecture capable of executing all three operations within a unified datapath. By exploiting common computational patterns across LayerNorm, RMSNorm and Softmax the proposed vector engine maximizes hardware sharing while reducing implementation overhead. Physical ASIC implementation results show that MIVE provides comprehensive multi-function support while achieving higher area and hardware efficiency than most state-of-the-art standalone accelerators.
