Search papers, labs, and topics across Lattice.
This paper introduces LAST, a novel framework for query-dependent visual token pruning in edge-cloud collaborative multimodal large language model (MLLM) inference, which addresses the inefficiencies of existing pruning methods that are either query-agnostic or reliant on cloud model access. By leveraging a compact edge-side vision-language model (VLM) to derive importance signals from the last query token's attention, LAST enables efficient pruning of visual tokens without incurring the costs of autoregressive generation or attention aggregation. Experimental results demonstrate that LAST retains 95.4% of full-token accuracy while reducing the visual token count to just 12.5%, significantly lowering cloud-side computation costs.
LAST achieves 95.4% accuracy with only 12.5% of visual tokens, revolutionizing edge-cloud inference efficiency.
Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge devices encode visual inputs into tokens for a general-purpose cloud MLLM. However, dense visual-token sequences increase cloud-side inference costs. Existing pruning methods mainly target centralized inference: vision-driven methods can operate before cloud execution but are typically query-agnostic, whereas query-guided methods often rely on internal states of the target MLLM and cannot determine token relevance before transmission. Compact guidance models offer an alternative, but existing designs may require costly attention aggregation or auxiliary generation. We propose LAST, a training-free framework for query-dependent visual token pruning in edge-cloud collaborative MLLM inference. LAST uses a compact edge-side VLM as a guidance proxy and derives a lightweight importance signal from the last query token's attention to visual tokens. Under causal attention, the last query token can attend to the full visual sequence and the entire query context, enabling query-aware pruning without cloud-model access, autoregressive generation, or costly aggregation over multiple query positions. LAST then retains a diverse set of query-relevant visual tokens under a fixed token budget. We evaluate LAST on 11 multimodal benchmarks under multiple token budgets against pruning methods with different guidance strategies. Experiments show that LAST consistently achieves the strongest performance, preserving 95.4% of the full-token accuracy while retaining only 12.5% of the visual tokens, with low edge-side selection overhead and reduced cloud-side computation.
