Search papers, labs, and topics across Lattice.
This paper introduces MS-Resampler, a multi-scope visual resampling framework designed to enhance the efficiency of multimodal large language models (MLLMs) by employing multiple scope-specific resamplers that leverage spatial scope priors. By allowing for the aggregation of visual information at varying granularities, from local details to global context, MS-Resampler significantly improves visual understanding and multimodal reasoning compared to traditional single-scope methods. Experimental results across ten public benchmarks demonstrate that this approach not only enhances performance but does so with minimal additional computational cost.
MS-Resampler boosts multimodal reasoning capabilities while maintaining efficiency, outperforming traditional methods with minimal overhead.
Multimodal large language models (MLLMs) typically employ resampling-based projectors to transform dense visual features into a compact token sequence for language modeling. Most existing resamplers adopt a single, fixed aggregation scope via global cross-attention, which can blur fine-grained local evidence and limit the ability to capture both local details and global context within a fixed token budget. In this work, we propose MS-Resampler, a multi-scope visual resampling framework for MLLMs. MS-Resampler instantiates multiple scope-specific resamplers by injecting explicit spatial scope priors into the resampling attention, enabling each branch to aggregate visual information at a particular granularity from local to global. The outputs of these scope-specific resamplers are then adaptively fused to produce the final visual representations for language modeling. Extensive experiments on ten public multimodal benchmarks show that MS-Resampler consistently improves visual understanding and multimodal reasoning over conventional single-scope resamplers, while introducing only minimal computational overhead.
