Search papers, labs, and topics across Lattice.
This paper explores the design space of lookup table (LUT)-based hardware architectures for ternary weight quantization in large language model (LLM) inference, addressing the inefficiencies of conventional compute platforms that lack native support for ternary-weight arithmetic. By introducing an open-source hardware generator and an analytical cost model, the authors enable systematic design space exploration and fair cross-design evaluation, revealing critical insights about architectural trade-offs. Key findings indicate that optimal architecture is determined by the activation data type, with significant area reductions achieved鈥攗p to 2.2x compared to multiplier-based designs鈥攂y maximizing core size and correcting suboptimal parameters.
LUT-based hardware architectures can achieve up to 2.2x area reduction for LLM inference by challenging conventional design assumptions and optimizing for activation data types.
Ternary weight quantization (e.g., BitNet b1.58) offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. However, conventional compute platforms lack native support for ternary-weight arithmetic, often relying on inefficient dequantization. Lookup table (LUT)-based hardware architectures provide an effective alternative by replacing multiplications with conditional additions, but their design space remains largely unexplored. Existing designs rely on heuristic parameter selection, lacking a systematic understanding of the architectural trade-offs. This work addresses this gap by formalizing the design space of ternary LUT-based accelerators and presenting an open-source hardware generator coupled with an analytical cost model, validated against synthesis in TSMC 16nm technology. By spanning the full architectural space, this framework not only enables rapid design space exploration but also establishes a common footing for fair cross-design evaluation, which was previously hindered by inconsistent instantiations across published accelerators. Using this framework, we challenge several assumptions and design choices in recent literature. We demonstrate that the optimal architecture is fundamentally governed by the activation data type: while LUT-based reuse offers significant gains for high-cost arithmetic (e.g., FP16), it yields diminishing returns for small integer types. Furthermore, we show that maximizing core size consistently improves area density compared to highly tiled approaches. Our optimized designs achieve a 2.2x area reduction compared to multiplier-based baselines. Moreover, by benchmarking state-of-the-art implementations against our model, we reveal that correcting suboptimal parameters yields up to a 1.2x area improvement.
