Search papers, labs, and topics across Lattice.
This work characterizes MXFP dot product implementations on Altera Agilex-5 FPGAs, revealing that while tensor mode excels for MXFP4 and MXFP6, it lacks support for MXFP8 and certain MXFP6 precisions. To address this limitation, the authors propose modifications to the DSP block's architecture, enabling native support for all MXFP formats while maintaining backward compatibility. The enhanced DSP block design achieves a 4.2x average throughput improvement across all MXFP formats with a modest area increase of 36% in the DSP tile.
Achieving a 4.2x throughput boost for MXFP formats could redefine efficiency benchmarks in low-precision deep learning inference on FPGAs.
Modern deep learning workloads increasingly rely on narrow numerical formats to improve efficiency and reduce memory footprint. The recently standardized microscaling floating-point (MXFP) family of formats, including MXFP8, MXFP6, and MXFP4, offers a practical approach to low-precision inference, yet the digital signal processing (DSP) blocks in current FPGA architectures offer limited native support for these formats. In this work, we first present a comprehensive characterization of MXFP dot product implementations on Altera Agilex-5 FPGAs, exploring a range of strategies spanning pure soft logic, DSP blocks in fixed-point, floating-point, and tensor modes. Our results show that while the tensor mode delivers the highest arithmetic density for MXFP4 (E2M1) and MXFP6 (E2M3), it cannot implement MXFP6 (E3M2) or any MXFP8 precisions, forcing designers to fall back to lower-density alternatives. Motivated by this gap, we propose targeted modifications to the DSP block's internal tensor-mode architecture that enable native support for all MXFP precisions while retaining backward compatibility. We estimate the area cost of these modifications using a simplified version of the Agilex-5 DSP block core implemented using the open-source ASAP7 PDK. We evaluate a variety of modified DSP block designs that present a tradeoff between format coverage, arithmetic density, and area overhead. Our preferred design point increases the DSP tile area by 36%, corresponding to only 1.8\% of the total FPGA die area. We evaluate the device-level impact of our enhanced DSP block by comparing systolic array matrix multiplier implementations across all MXFP precisions, contrasting the best-available strategies on the existing architecture against designs leveraging our modified DSP block. Our results demonstrate an average throughput improvement of 4.2x across all supported MXFP formats.
