Search papers, labs, and topics across Lattice.
This paper introduces MxGLUT, a reconfigurable LUT-centric broadcast dataflow accelerator designed to optimize mixed-precision GEMM operations for large language model inference. By unifying FP8-INT4 and FP8-FP8 computations within a single LUT-based execution framework and employing a novel RLB dataflow, MxGLUT significantly reduces hardware redundancy and enhances efficiency during both prefill and decode phases. The results demonstrate up to 2.16x latency speedup and substantial power savings, making it a compelling solution for improving the performance of LLMs under weight-only quantization scenarios.
MxGLUT achieves up to 2.16x latency speedup and reduces energy consumption significantly while maintaining competitive perplexity levels in LLM inference.
Large language model (LLM) inference suffers from growing inefficiency across the prefill and decode phases, especially under weight-only quantization, where activations remain in FP8 while weights are compressed to low-bit integers. Existing LUT-based accelerators mainly target FP8-INT4 computation and still rely on separate floating-point (FP) datapaths for attention GEMM operations, leading to redundant hardware and non-unified mixed-precision execution. Moreover, their static dataflows are poorly matched to the distinct prefill and decode phases. To address these challenges, we propose MxGLUT, a reconfigurable LUT-centric broadcast (RLB) dataflow accelerator built on mixed-precision LUT-based processing elements (MxLPEs). Guided by a unified LUT-based execution framework, MxGLUT organizes both FP8-INT4 and FP8-FP8 GEMMs under a single LUT-based compute mechanism without dedicated FP multipliers or additional FP datapaths, and further adopts the RLB dataflow that localizes heavy partial-sum accumulation during the prefill phase and exploits weight reuse in the decode phase. Synthesized in UMC $28\,\mathrm{nm}$ CMOS at $200~\mathrm{MHz}$, MxGLUT reduces multiplier area by up to $56.92\%$ and power by up to $77.07\%$ and $78.35\%$ in FP8-INT4 and FP8-FP8 modes, respectively. At the accelerator level, MxGLUT achieves an area efficiency of $0.492~\mathrm{TFLOPS/mm^2}$ and an energy efficiency of $11.58~\mathrm{TFLOPS/W}$, while adding native FP8-FP8 support incurs only $2.57\%$ and $3.34\%$ reductions in area and energy efficiency, respectively, relative to the FP8-INT4-only FIGLUT baseline. Across the Llama family, MxGLUT achieves up to $2.16\times$ and $1.49\times$ latency speedup, and reduces normalized energy to $0.44\times$ and $0.71\times$ in prefill and decode, respectively, with at most $1.70\%$ perplexity increase.
