Search papers, labs, and topics across Lattice.
SPAR distills the spatial reasoning of a finely-strided, sliding-window Vision Transformer (ViT) teacher into a single-pass student network. This is achieved via feature regression, enabling the student to perform efficient, high-resolution inference without architectural changes or pixel-level supervision. Applied to open-vocabulary segmentation, SPAR significantly outperforms single-pass baselines and even surpasses the teacher model in mIoU.
Achieve state-of-the-art open-vocabulary segmentation by distilling a sliding-window ViT into a single-pass architecture, enabling efficient high-resolution reasoning.
Foundational Vision Transformers (ViTs) have limited effectiveness in tasks requiring fine-grained spatial understanding, due to their fixed pre-training resolution and inherently coarse patch-level representations. These challenges are especially pronounced in dense prediction scenarios, such as open-vocabulary segmentation with ViT-based vision-language models, where high-resolution inputs are essential for accurate pixel-level reasoning. Existing approaches typically process large-resolution images using a sliding-window strategy at the pre-training resolution. While this improves accuracy through finer strides, it comes at a significant computational cost. We introduce SPAR: Single-Pass Any-Resolution ViT, a resolution-agnostic dense feature extractor designed for efficient high-resolution inference. We distill the spatial reasoning capabilities of a finely-strided, sliding-window teacher into a single-pass student using a feature regression loss, without requiring architectural changes or pixel-level supervision. Applied to open-vocabulary segmentation, SPAR improves single-pass baselines by up to 10.5 mIoU and even surpasses the teacher, demonstrating effectiveness in efficient, high-resolution reasoning. Code: https://github.com/naomikombol/SPAR
