Search papers, labs, and topics across Lattice.
This paper introduces SepPrune, a novel pruning framework that leverages separator tokens to efficiently reduce the number of vision tokens in multimodal large language models (MLLMs) without requiring architectural modifications. By utilizing attention scores that peak at these separator tokens, SepPrune ranks and selects the most informative vision tokens, achieving significant reductions in computational cost. Experimental results on Qwen2.5-VL-7B demonstrate that SepPrune maintains 96.3% of the original model's accuracy while pruning 80.2% of the vision tokens, marking a substantial advancement in MLLM efficiency.
Pruning 80.2% of vision tokens without sacrificing accuracy could revolutionize the efficiency of multimodal large language models.
Recent multimodal large language models (MLLMs), such as Qwen2.5-VL and InternVL3, generate large numbers of vision tokens for high-resolution inputs, leading to substantial computational cost. Existing vision token pruning methods either depend on cross-modal attention and cannot prune before the prefill stage, or rely on diversity estimation with high computational overhead. We observe that attention scores from both vision and text tokens peak at modality separator tokens, suggesting that these separators bridge the two modalities. Based on this observation, we propose SepPrune, an efficient, training-free, plug-and-play pruning method that uses the separator token as a unified query to rank and select informative vision tokens. SepPrune reuses the LLM's built-in projection parameters and requires no architectural changes. Experiments on Qwen2.5-VL-7B show that SepPrune achieves state-of-the-art performance, retaining 96.3% of the original accuracy while removing 80.2% of vision tokens.
Tsinghua AI