Search papers, labs, and topics across Lattice.
This paper implements and evaluates an energy-efficient Whisper dot-product kernel offloading on the IMAX architecture, targeting the significant computational bottleneck of dot-product operations in Whisper models. By employing techniques such as kernel mapping, local-memory sizing, and burst scheduling, the authors achieve a substantial reduction in power-delay product (PDP) and energy-delay product (EDP) for Whisper-tiny.en. The results indicate that the IMAX architecture can achieve a PDP of 11.58J, outperforming leading platforms like Jetson AGX Orin and RTX 4090 by factors of 2.35x and 10.48x, respectively.
Whisper models can achieve unprecedented energy efficiency on the IMAX architecture, slashing power consumption by over 10x compared to top GPUs.
In this paper, we implement and evaluate Whisper dot-product kernel offloading on IMAX, a programmable Coarse-Grained Linear Arrays (CGLAs) architecture. Whisper-tiny.en profiling on an ARM Cortex-A72 shows that dot-product operations account for 90.6% of FP16 execution time and 87.1% of Q8_0 execution time. To address this kernel bottleneck, we combine kernel mapping, local-memory sizing, and burst scheduling. The implementation uses inline FP16-to-FP32 conversion, 2-way SIMD FMA on a 64-bit datapath, column-wise multithreading, and mixed execution in which aligned vector segments run on IMAX and residual segments run concurrently on the host CPU. We evaluate the design with an FPGA prototype and a 28nm ASIC projection at 840MHz. For Whisper-tiny.en, 32KB local memory and burst length 16 jointly minimize PDP and EDP. Under a TDP-based cross-platform comparison, the projected IMAX records a PDP of 11.58J for Whisper-tiny.en Q8_0, 2.35x lower than Jetson AGX Orin (27.16J) and 10.48x lower than RTX 4090 (121.38J). The same design extends to Whisper-base.en and Whisper-small.en, where the PDP gap narrows as 32KB local-memory coverage drops from 93.8% for tiny to about 66.5% for base and small. These results position IMAX as a programmable architecture for lower-PDP local ASR in the tiny-model regime.
