Search papers, labs, and topics across Lattice.
This paper introduces GSTEP, a novel token pruning framework for VideoLLMs that addresses the inefficiencies of existing local pruning methods by modeling video as a continuous spatio-temporal information flow. By combining continuous temporal density with intra-frame spatial density, GSTEP enables global token sampling that maintains critical information while significantly reducing redundant tokens. Experimental results show that GSTEP can prune 75% of visual tokens while preserving up to 100.2% of original performance and achieving a 1.17x speedup on LLaVA-OneVision-7B.
Pruning 75% of visual tokens without sacrificing performance could redefine efficiency benchmarks for VideoLLMs.
Video large language models (VideoLLMs) achieve strong video understanding performance, but their inference remains expensive due to the large number of redundant spatio-temporal visual tokens in long videos. Existing token pruning methods alleviate this cost by reducing redundant tokens, yet most of them rely on segment-level local pruning, where videos are partitioned into isolated segments and tokens are selected independently within each segment. Such designs may under-preserve short but semantically dense segments and discard tokens that appear non-salient locally but remain critical from a global perspective. To address this issue, we propose GSTEP (Global Spatio-Temporal Density Pruning), a plug-and-play pruning framework that models video as a continuous spatio-temporal information flow. GSTEP constructs a token-level spatio-temporal density by combining a continuous temporal density, obtained from a smoothed centered frame-level change signal, with intra-frame spatial density, and then performs global token sampling by jointly balancing information density and coverage. Extensive experiments on multiple VideoLLMs and public benchmarks demonstrate that GSTEP consistently achieves strong accuracy-efficiency trade-offs and generalizes well across model architectures and evaluation settings. On LLaVA-OneVision-7B, GSTEP prunes 75% of visual tokens, preserves up to 100.2% of the original average performance across benchmarks, and achieves a 1.17 end-to-end speedup.
