Search papers, labs, and topics across Lattice.
This paper introduces InfoMerge, a training-free method for compressing visual tokens in Video Large Language Models (Video-LLMs) that enhances efficiency while preserving performance. By employing a novel Temporal Fingerprint Difference strategy for redundancy estimation and a Content-Aware Budget Allocation (CABA) approach for dynamic token distribution, InfoMerge effectively addresses the limitations of existing compression techniques that struggle with frame-level noise and non-uniform information distribution. Experimental results demonstrate that InfoMerge can reduce visual tokens by 85% while maintaining 98.8% of original performance, achieving a 4.24-fold speedup in the prefill stage across various benchmarks.
InfoMerge achieves an 85% reduction in visual tokens while retaining nearly all original performance, revolutionizing efficiency in Video-LLMs.
Video Large Language Models (Video-LLMs) achieve strong performance in video understanding, but their excessive visual tokens bring substantial computational overhead. Existing training-free compression methods improve inference efficiency by reducing visual tokens, yet they often rely on local adjacent-frame similarity for temporal redundancy estimation or allocate token budgets mainly according to segment length. Such designs are sensitive to frame-level noise and fail to capture the non-uniform information distribution of real-world videos. To address these challenges, we propose InfoMerge, a training-free visual token compression method that improves token utilization through robust redundancy estimation and content-aware budget allocation. Specifically, we propose the Temporal Fingerprint Difference: a segment-level second-order temporal redundancy estimation strategy, which models the temporal similarity structure of tokens at the same spatial positions within each segment. We further introduce Content-Aware Budget Allocation (CABA), which dynamically allocates segment-level token budgets based on segment uniqueness and spectral-entropy-based representational richness. By reducing repeated preservation of redundant static regions and allocating more tokens to informative segments, InfoMerge makes better use of the limited token budget while maintaining strong performance. Extensive experiments show that InfoMerge achieves strong efficiency--accuracy trade-offs across multiple benchmarks and backbones, with more pronounced advantages under aggressive compression. On LLaVA-OneVision-7B, InfoMerge retains 98.8\% of the original average performance while reducing 85\% of visual tokens and achieving a 4.24-fold speedup in the prefill stage.
