Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DeVIT, an innovative acceleration method for vision transformers that utilizes delta computation to facilitate multiplier-less matrix multiplication, significantly reducing computational complexity. By focusing on low-bit model weights and the concept of value locality, DeVIT enhances efficiency while maintaining performance on resource-constrained devices. The key result demonstrates that this approach can effectively lower both processing and memory demands, making transformer models more viable for deployment in low-power environments.
Delta computation enables vision transformers to achieve high efficiency without multipliers, unlocking new possibilities for low-power AI applications.
The emergence of transformer-based deep learning models has brought unprecedented performance across various domains, particularly in natural language processing and computer vision. However, deploying these models, especially on resource-constrained devices, poses significant challenges due to their high computational complexity and large memory size and bandwidth requirements. This complexity has led researchers to use low-bit model weights to reduce memory usage and improve efficiency. In addition to reducing processing and memory demands, quantization introduces another useful property: value locality, where the extremely large number of parameters are restricted to a limited range of values. To fully take advantage of this locality, this paper presents DeVIT, an acceleration method for vision transformers that leverages differential computation to enable multiplier-less matrix multiplication.