Search papers, labs, and topics across Lattice.
This paper introduces W16A16, a novel 16-bit quantization scheme for deploying deep neural networks on microcontroller units (MCUs) that achieves significantly lower quantization errors than 8-bit methods while maintaining competitive inference speed and energy efficiency. By analyzing the Armv7E-M architecture, the authors demonstrate that 16-bit precision can yield up to 10x reduction in regression and classification errors with comparable or improved performance metrics. This work provides a practical approach to high-accuracy edge AI inference on resource-constrained devices.
Achieve 10x lower quantization errors on MCUs with 16-bit precision, matching or beating 8-bit inference speed and energy costs.
To deploy deep neural networks on edge hardware, highly efficient inference schemes are necessary that retain high accuracy. This work presents W16A16, a high precision (16-bit), fast speed, low energy quantization method. On a widely applied microcontroller architecture Armv7E-M, our proposed approach achieves faster speed and lower energy consumption on layer- and model-level compared to alternative quantization schemes. We analyze the architecture of Armv7E-M, explain the underlying principles behind the performance advantages of 16-bit approaches, and evaluate the empiric quantization errors for regression and classification tasks, as well as empiric time- and energy consumption in MCU deployment. We observe ca.\ 10 times lower quantization errors compared to 8-bit quantization schemes while achieving similar or better inference times and energy consumption.
