Search papers, labs, and topics across Lattice.
OccamToken is introduced as a training-free framework for efficient vision-language model (VLM) inference by pruning visual tokens. It addresses the limitations of absolute-ranking pruning methods by using register-anchored relative evidence testing, evaluating tokens based on their information contribution beyond a register-based reference. The method achieves significant visual token compression (e.g., reducing 2,880 tokens to ~40) while maintaining high accuracy across multiple VLM architectures like LLaVA-NeXT, LLaVA-v1.5, and Qwen3-VL.
Ditch brittle token rankings: OccamToken uses register-anchored relative evidence testing to prune visual tokens in VLMs, achieving extreme compression (down to 1.4%!) without retraining or significant accuracy loss.
Vision-language models (VLMs) rely on long visual token sequences for visual understanding, making the prefill stage expensive in both computation and memory. Most existing pruning methods follow an absolute-ranking paradigm, assigning importance scores to visual tokens and retaining a fixed top-K subset. In this work, we argue that this paradigm is fundamentally brittle: attention sinks distort token importance rankings, while image redundancy and query-dependent visual evidence make fixed token budgets unreliable across inputs. We propose OccamToken, a training-free framework that replaces absolute token ranking with register-anchored relative evidence testing. Instead of asking which tokens are globally important, OccamToken evaluates whether a visual token provides information beyond a register-based reference. Our key insight is that register tokens naturally absorb low-information attention patterns, making them a stable reference for identifying genuinely informative visual evidence. Based on this principle, OccamToken performs both image-adaptive redundancy pruning and query-adaptive relevance pruning through dynamic thresholds derived from register attention. Across LLaVA-NeXT, LLaVA-v1.5, and Qwen3-VL, OccamToken consistently improves the accuracy-efficiency trade-off without additional training. Notably, on LLaVA-NeXT, it reduces 2,880 visual tokens to approximately 40 while preserving over 93% of the original accuracy, enabling stable visual token compression even in the extreme 1.4% retention regime.
