Search papers, labs, and topics across Lattice.
This paper introduces the Euclidean enhanced Vision Transformer (EVT), which improves upon the existing Vision Transformer (ViT) architecture by integrating explicit spatial priors through a more effective Euclidean distance decay mechanism. By replacing the decomposed attention method of its predecessor RMT with a spatially-independent grouping approach, EVT enhances flexibility in token management while maintaining computational efficiency. Extensive experiments show that EVT achieves a remarkable 86.6% top-1 accuracy on ImageNet-1k without requiring additional training data, indicating its superior performance across various computer vision tasks.
EVT achieves 86.6% top-1 accuracy on ImageNet-1k without extra training data, redefining the potential of Vision Transformers in computer vision.
In recent years, the Vision Transformer (ViT) has garnered significant attention within the computer vision community. However, the core component of ViT, Self-Attention, lacks explicit spatial priors and suffers from quadratic computational complexity, limiting its applicability. To address these issues, we have proposed RMT, a robust vision backbone with explicit spatial priors for general purposes. RMT utilizes Manhattan distance decay to introduce spatial information and employs a horizontal and vertical decomposition attention method to model global information. Building on the strengths of RMT, Euclidean enhanced Vision Transformer (EVT) is an expanded version that incorporates several key improvements. Firstly, EVT uses a more reasonable Euclidean distance decay to enhance the modeling of spatial information, allowing for a more accurate representation of spatial relationships compared to the Manhattan distance used in RMT. Secondly, EVT abandons the decomposed attention mechanism featured in RMT and instead adopts a simpler spatially-independent grouping approach, providing the model with greater flexibility in controlling the number of tokens within each group. By addressing these modifications, EVT offers a more sophisticated and adaptable approach to incorporating spatial priors into the Self-Attention mechanism, thus overcoming some of the limitations associated with RMT and further enhancing its applicability in various computer vision tasks. Extensive experiments on Image Classification, Object Detection, Instance Segmentation, and Semantic Segmentation demonstrate that EVT exhibits exceptional performance. Without additional training data, EVT achieves 86.6% top1-acc on ImageNet-1k.
