Search papers, labs, and topics across Lattice.
This paper introduces OmniDelta, a skill-driven framework designed to optimize budget allocation for token compression in Omni-modal Large Language Models (OmniLLMs). By addressing the limitations of existing methods that rely on fixed budgets and uniform allocation, OmniDelta dynamically adjusts token retention based on query intent and content relevance, leading to improved efficiency. Experimental results demonstrate that OmniDelta achieves a new accuracy-efficiency Pareto frontier, significantly reducing memory usage and increasing inference speed while maintaining performance on audio-video benchmarks.
OmniDelta achieves a 1.64x speedup in inference while reducing GPU memory usage by 22% at just 25% token retention, redefining efficiency in OmniLLMs.
Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory and inference costs. Existing compression methods mainly focus on selecting important tokens under fixed budgets, leaving the preceding budget-allocation problem underexplored. We show that direct query-to-audio/video similarity is unreliable for inter-modal budget allocation, and that uniform intra-modal budgets can miss key evidence while retaining redundant content. To address these limitations, we propose OmniDelta, a training-free, skill-driven framework that couples intent-aware inter-modal allocation with content-aware intra-modal allocation. OmniDelta first constructs audio and video skill pools to shift the fixed retained-token budget according to query demand, then reallocates modality budgets over audio segments and video frames using local complexity and temporal redundancy. The resulting local budgets can be combined with existing pruning strategies, preserving the total retained-token ratio while changing where the budget is spent. Experiments on four audio-video benchmarks with two Qwen2.5-Omni models show that OmniDelta establishes a new accuracy-efficiency Pareto frontier across pruning ratios. At 25% token retention on Qwen2.5-Omni-7B, OmniDelta reduces GPU memory by 22.0% and achieves a 1.64x end-to-end speedup over full-token inference.
DAMO