Search papers, labs, and topics across Lattice.
This paper introduces REDI, a novel method for optimizing token allocation in Vision Transformers by leveraging class-conditioned corpus scores derived from supervised TF-IDF. By quantizing DINOv3 patch representations and integrating attention maps, REDI effectively reduces the sequence length from 201 to 107 tokens while achieving a Top-1 accuracy of 84.706% on ImageNet-1K, surpassing traditional methods. The findings highlight the complementary benefits of combining corpus statistics and attention signals for improved patch ranking in image classification tasks.
REDI achieves a remarkable 46.8% reduction in token sequence length while boosting accuracy, showcasing the power of combining class-specific corpus statistics with attention mechanisms.
Most token reduction methods for Vision Transformers seek favorable tradeoffs between accuracy and efficiency by pruning, merging, or pooling patch tokens. REDI (Relevance for DINOv3 Token Reduction) studies this question through a controlled supervised reference: how should a fixed token budget be allocated across patches for image classification? REDI quantizes final block DINOv3 patch representations into a visual vocabulary and derives class conditioned corpus scores using supervised TF-IDF over visual words. For each validation image, the ground truth class selects a row of the TF-IDF table, and four transformed views produce a TF-IDF map aligned to a reference center crop. A separate dense pass on the same crop provides an attention map. After independent min max normalization, their elementwise product defines the REDI score. A fixed keep, merge, and compress operator then uses score rank to assign patch roles and score magnitude to weight merging and compression. With precomputed REDI scores, a frozen DINOv3 ViT-B/16 backbone, and the same linear classifier used for dense evaluation, the operator reduces the sequence length from 201 to 107 tokens, a 46.8% sequence reduction. The REDI variant based on incoming attention mass achieves 84.706% Top-1 accuracy on ImageNet-1K, compared with 83.514% for the dense baseline, 82.634% for incoming attention mass alone, and 81.796% for supervised TF-IDF alone. The same corpus term also improves reduced classification for three alternative attention formulations relative to their attention only counterparts. Together, these controlled comparisons indicate that class specific corpus statistics and image specific attention provide complementary signals for patch ranking in this setting.
