Search papers, labs, and topics across Lattice.
This paper introduces CIMERA, a reconfigurable-precision inference accelerator designed to address the high computational and memory demands of large language models (LLMs). By leveraging compute-in-interconnect and memory, CIMERA enables precision-aware execution that adapts to the heterogeneous workloads of LLMs, significantly improving energy efficiency. Experimental results show that CIMERA achieves up to 25x and 10x higher energy efficiency compared to the Nvidia H100 for 1B and 13B models, respectively, highlighting its potential for both data centers and edge devices.
CIMERA achieves up to 25x energy efficiency improvements for LLM inference, revolutionizing how we approach resource-constrained environments.
LLM impose significant computational and memory demands, creating challenges for energy-efficient inference across platforms ranging from data centers to power-constrained edge devices. Weight precision plays a critical role in balancing inference accuracy, throughput, and energy consumption, while modern LLM workloads exhibit pronounced heterogeneity and tolerance that favors adaptive precision execution. This paper presents CIMERA, a reconfigurable-precision LLM inference accelerator that integrates compute-in-interconnect and memory to mitigate the memory wall and enable precision-aware execution. Compared to Nvidia H100, CIMERA delivers up to $25\times$ and $10\times$ higher energy efficiency for 1B and 13B models, respectively.
