Search papers, labs, and topics across Lattice.
This paper introduces CoverPruner, a novel approach to visual token pruning that optimizes coverage by identifying which surviving tokens best represent removed tokens in vision-language models (VLMs). Unlike traditional methods that focus solely on which tokens to retain, CoverPruner employs Representational Coverage Maximization (RCM) to ensure that the essential information is preserved even after aggressive pruning. The results demonstrate that CoverPruner outperforms existing methods in accuracy across various VLM architectures, particularly under high compression rates, highlighting its effectiveness in maintaining performance while reducing inference costs.
CoverPruner reveals that optimizing for representational coverage can significantly enhance the accuracy of pruned vision-language models, especially when faced with aggressive compression.
Visual token pruning reduces the inference cost of vision-language models (VLMs), but most methods only ask which tokens to keep. This retained-token view can keep redundant high-scoring tokens while leaving discarded evidence without a close representative. We propose CoverPruner, a training-free pruner that asks the complementary demand-side question: after a token is removed, which surviving original token represents it for the target VLM? CoverPruner formulates pruning as Representational Coverage Maximization (RCM), covering the full projected visual-token set with query-weighted demand. It instantiates RCM with projector-space coverage and a lightweight first-layer attention probe. Across multiple VLM architectures and compression rates, CoverPruner achieves the best average accuracy among all compared methods, with the largest gains usually appearing under aggressive compression.
