Search papers, labs, and topics across Lattice.
This paper introduces MedARC, a training-free framework that enhances the efficiency of visual token processing in 3D medical vision-language models by employing a saliency-aware merging strategy. By integrating multiple cues鈥攕elf-attention from the VLM, similarity to text embeddings, and deviations from volume-level features鈥擬edARC effectively identifies and preserves clinically relevant tokens while reducing redundancy. Experimental results demonstrate that MedARC significantly decreases visual-token overhead and inference time, all while maintaining or improving diagnostic performance on CT-RATE and MR-RATE datasets.
MedARC's adaptive redundancy compression preserves critical diagnostic information while slashing visual-token overhead, revolutionizing efficiency in 3D medical VLMs.
Integrating 3D medical images with vision-language models (VLMs) holds substantial promise for computer-aided diagnosis. However, volumetric images generate prohibitively long visual-token sequences with considerable spatial and inter-slice redundancy. Existing token compression methods typically apply uniform reduction or rely on a single importance signal, increasing the risk of removing regions that are clinically relevant to the query or structurally distinctive. To address this limitation, we propose MedARC, a unified, training-free framework for Adaptive Redundancy Compression of visual tokens in 3D medical VLMs. MedARC estimates token importance by integrating three complementary cues: self-attention from the VLM vision encoder, which reflects the model's intrinsic visual focus; similarity between projected visual tokens and text embeddings, which identifies query-relevant regions; and deviations of local visual foundation model features from the volume-level feature center, which highlight structurally distinctive anatomy. The resulting importance distribution guides a saliency-aware merging strategy that preserves informative tokens while consolidating redundant ones rather than simply discarding them. Experiments on CT-RATE and MR-RATE show that MedARC reduces visual-token overhead and inference time while preserving or improving diagnostic performance. Its multi-cue scoring cost is outweighed by the savings from processing fewer tokens, with greater benefits expected for larger language models.
