Search papers, labs, and topics across Lattice.
This paper introduces OmniScope, a novel token compression framework for omnimodal large language models that addresses the limitations of existing methods by decoupling relevance estimation for audio and video based on a shared query. The authors demonstrate that their approach significantly improves performance by allowing modality-specific token budgets and employing an anchor-delta strategy to preserve critical information during aggressive compression. Results show that OmniScope achieves superior accuracy and efficiency, with up to 3.53x prefill speedup and over 15% GPU memory reduction at 25% token retention, while maintaining high accuracy levels.
OmniScope reveals that treating audio and video relevance separately can drastically enhance performance in omnimodal models, achieving remarkable efficiency gains without sacrificing accuracy.
Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.
DAMO