Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces VisionScreen, a novel vision model that applies the Screening mechanism from language modeling to visual recognition by evaluating the relevance of image patches based on query-key similarity. By allowing each patch to selectively aggregate only relevant content and spatially related patches, VisionScreen overcomes the limitations of traditional self-attention in Vision Transformers, which often include irrelevant background information. Experimental results on image classification benchmarks show that VisionScreen significantly outperforms conventional ViT models, highlighting the effectiveness of Screening in visual contexts.
VisionScreen redefines visual recognition by enabling selective aggregation of image patches, leading to superior performance over traditional Vision Transformers.
Vision Transformer (ViT) has been widely used as a powerful framework for modeling global dependencies among image patches. However, its core component, self-attention assigns softmax-normalized relative weights to all patches, making it difficult to evaluate the relevance between patches independently. In visual recognition, images often contain many background or redundant patches, yet self-attention cannot explicitly reject such irrelevant patches, which may introduce unnecessary information into feature aggregation. To address this limitation, Screening has been proposed in the field of language modeling, where the relevance of each token is independently evaluated based on query-key similarity and low-relevance tokens are explicitly excluded through thresholding. In this work, we propose VisionScreen, a new vision model that extends Screening mechanism to visual recognition. VisionScreen treats image patches as tokens arranged on a two-dimensional grid and extends absolute relevance estimation based on query-key similarity to the two-dimensional spatial domain. This allows each patch to selectively aggregate only content-wise and spatially relevant patches without relying on competition among patches. Experiments on image classification benchmarks demonstrate that the proposed method outperforms conventional ViT. These results suggest that Screening can be effective for visual recognition, offering an alternative to relative feature aggregation based on softmax attention.