Search papers, labs, and topics across Lattice.
DIPrune addresses semantic degradation in training-free MLLM pruning by reformulating it as task loss minimization and deriving a token-wise upper bound. This formulation reveals an inter-layer gradient term, leading to DIPrune's dual importance scoring mechanism that balances intra-layer feature saliency and inter-layer semantic evolution. Experiments on LLaVA and Qwen-VL show DIPrune achieves state-of-the-art performance, preserving crucial signals for deep reasoning.
LLM pruning can be task-aware: DIPrune's dual importance scoring jointly optimizes static feature saliency and dynamic semantic evolution, outperforming prior methods on LLaVA and Qwen-VL.
Recent training-free pruning approaches for Multimodal Large Language Models (MLLMs) effectively cut computational overhead by exploiting visual redundancy or text-vision attention. However, they frequently suffer from semantic degradation due to their task-agnostic design or unreliable attention estimates. Based on our empirical analysis, we have found that this issue arises because salient tokens in shallow layers persistently suppress emerging semantic ones through numerical inertia, leading to premature discarding of signals crucial for deep reasoning. To address the aforementioned issue, from the task-oriented aspects, we first reformulate training-free pruning as a minimization of the distortion in the final task loss and derive a tractable, token-wise upper bound to serve as a surrogate objective. Specifically, this formulation inherently reveals a previously neglected inter-layer term that accounts for gradients across layers. Accordingly, for the implementation, we propose DIPrune, a rank-based framework that employs a dual importance scoring mechanism to jointly optimize intra-layer static feature saliency and inter-layer dynamic semantic evolution. Extensive experiments on LLaVA and Qwen-VL demonstrate that DIPrune consistently achieves state-of-the-art results.
