Search papers, labs, and topics across Lattice.
This paper introduces CIMERA, a reconfigurable-precision inference accelerator designed to address the high computational and memory demands of large language models (LLMs). By integrating compute-in-interconnect and memory, CIMERA enables precision-aware execution that adapts to the heterogeneity of LLM workloads, significantly improving energy efficiency. The results show that CIMERA achieves up to 25x and 10x higher energy efficiency compared to the Nvidia H100 for 1B and 13B models, respectively, highlighting its potential for both data centers and edge devices.
CIMERA achieves up to 25x higher energy efficiency for LLM inference by leveraging adaptive precision execution, challenging the status quo of traditional accelerators.
LLM impose significant computational and memory demands, creating challenges for energy-efficient inference across platforms ranging from data centers to power-constrained edge devices. Weight precision plays a critical role in balancing inference accuracy, throughput, and energy consumption, while modern LLM workloads exhibit pronounced heterogeneity and tolerance that favors adaptive precision execution. This paper presents CIMERA, a reconfigurable-precision LLM inference accelerator that integrates compute-in-interconnect and memory to mitigate the memory wall and enable precision-aware execution. Compared to Nvidia H100, CIMERA delivers up to $25\times$ and $10\times$ higher energy efficiency for 1B and 13B models, respectively.
