Search papers, labs, and topics across Lattice.
This paper introduces Trend-aware Pruning, a novel framework for token pruning in Multimodal Large Language Models (MLLMs) that addresses the limitations of existing training-free methods by modeling token importance as a dynamic trajectory rather than a static snapshot. By capturing the momentum of attention flow, the framework allows for the selective reactivation of "late-blooming" tokens that gain semantic importance over time, preventing the premature loss of critical information. Experimental results show that this approach achieves a remarkable efficiency-performance trade-off, reducing visual tokens by over 77.8% while maintaining competitive performance across various multimodal tasks.
"Late-blooming" tokens can be crucial for deep-layer reasoning, and our method ensures they aren't discarded prematurely, achieving over 77.8% token reduction without sacrificing performance.
While visual token pruning is essential for efficient Multimodal Large Language Models (MLLMs), existing training-free methods suffer from a critical limitation: they rely on static, instantaneous heuristics to perform irreversible filtering. This approach ignores the hierarchical nature of MLLMs, where token importance often evolves dynamically rather than remaining fixed across layers. Consequently, tokens essential for deep-layer reasoning are often prematurely discarded by shallow-layer estimates. To address this, we propose Trend-aware Pruning, a novel framework that elevates pruning from a local snapshot decision to a temporal trajectory modeling problem. Instead of relying on isolated scores, our method captures the momentum of attention flow. This enables a dynamic rectification mechanism that selectively reactivates"late-blooming"tokens, those initially undervalued but exhibiting rising semantic importance, thereby preventing the loss of critical visual cues. Extensive experiments demonstrate that our approach achieves a superior efficiency-performance trade-off across diverse multimodal tasks. Notably, it reduces visual tokens by over 77.8%, retaining only approximately 23 tokens in the final layer while maintaining competitive performance, offering a robust and reversible solution for high-efficiency multimodal inference.
