Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SpiralFovea, an innovative input-adaptive tokenization method that optimizes the representation of visual data by dynamically adjusting token identity, location, scale, and count based on local visual entropy. By replacing the conventional fixed-grid tokenization with a multi-scale spiral ring approach, SpiralFovea achieves significant improvements in accuracy and efficiency, yielding up to 2.1 percentage points in accuracy while reducing input tokens by 60% and self-attention FLOPs by 84%. The findings highlight that the effectiveness of this adaptive tokenization is particularly pronounced in self-supervised foundation models, suggesting a new avenue for enhancing model performance without increasing computational costs.
SpiralFovea reveals that adaptive tokenization can enhance model accuracy by over 2% while slashing input tokens and computational costs, challenging the status quo of fixed-grid approaches.
Most adaptive-inference techniques for foundation models change what the model does - early exit, MoE routing, KV-cache compression, dynamic attention sparsity. The input that hits the backbone, however, remains a fixed-grid tokenisation indifferent to image content. We argue that this is a missed lever. We present SpiralFovea, a parameter-free, input-adaptive tokeniser in which token identity, location, scale, and count are all functions of local visual entropy and selection completes before any backbone parameter is queried. Around content-driven hotspot anchors, multi-scale spiral rings produce <= 78 patches that replace the standard 196-patch ViT grid at the input stage. Across four canonical fine-grained benchmarks, SpiralFovea yields +1.7-2.1 pp accuracy with a 60% reduction in input tokens, an 84% reduction in self-attention FLOPs at every transformer layer, and 18-29% throughput gains over the matched static tokenisation baseline. A controlled ablation on CUB-200-2011 Genus across four backbones reveals a clean diagnostic: the gain magnitude tracks inversely with the strength of the backbone's whole-image positional prior, isolating self-supervised foundation models as the regime where input-adaptive tokenisation is most valuable.