Search papers, labs, and topics across Lattice.
This paper introduces A-PACK, a two-stage framework for optimizing omni-modal LLMs by deferring audio pruning until query-conditioned multimodal interactions arise. The authors demonstrate that audio tokens contain higher task-relevant information density compared to video, allowing for more effective compression strategies that leverage local audio-visual dynamics. As a result, A-PACK achieves superior performance across multiple benchmarks while significantly reducing prefill FLOPs by up to 78% and enhancing decoding throughput by up to 2.21x.
A-PACK reveals that deferring audio pruning can lead to a 78% reduction in prefill costs while boosting performance in omni-modal LLMs.
Omni-modal LLMs jointly process audio, video, and text, but long multimodal sequences incur substantial prefill and KV-cache costs. Existing omni-modal compression methods primarily focus on pre-LLM token reduction, leaving modality-specific compression across the LLM boundary underexplored. We propose A-PACK, a two-stage framework that defers audio pruning until query-conditioned multimodal interactions emerge. Our analysis shows that audio exhibits higher task-relevant information density and representational diversity per token than video. We further find that local audio-visual dynamics provide a more effective cue for visual selection than token-wise matching. We therefore preserve audio and compress video with local dynamics before the LLM, then progressively prune low-relevance audio and visual tokens and their KV-cache entries inside the LLM. Across four benchmarks on Qwen2.5-Omni-7B/3B, A-PACK achieves the strongest average performance among the evaluated prior methods while reducing prefill FLOPs by up to 78% and improving decoding throughput by up to 2.21x.
Amazon Science