Search papers, labs, and topics across Lattice.
MuCRASP is introduced as a structured pruning framework tailored for vision-language models (VLMs) to preserve chain-of-thought (CoT) reasoning accuracy during compression. It addresses the limitations of existing pruning methods by targeting reasoning-critical components, preserving cross-modal alignment, and accounting for layer-wise sensitivity. Experiments across multiple VLMs and reasoning benchmarks demonstrate that MuCRASP consistently preserves reasoning quality under increasing compression, achieving superior performance compared to existing pruning approaches.
Pruning VLMs doesn't have to destroy chain-of-thought reasoning – MuCRASP shows how to surgically remove parameters while maintaining reasoning consistency and even outperforming strong baselines.
Vision-language models (VLMs) increasingly rely on chain-of-thought (CoT) reasoning to solve complex multimodal tasks, but their large parameter sizes make deployment expensive. Structured pruning offers a natural solution; however, existing methods fail to preserve CoT reasoning accuracy in VLMs. We identify two key reasons: (1) CoT consistency depends on sparse transition points (pivot tokens) in the generation trajectory, while existing pruning methods are CoT-agnostic; and (2) pruning methods designed for unimodal LLMs do not account for activation-distribution differences across visual and textual modalities. Motivated by these observations, we propose MuCRASP, a structured pruning framework that targets reasoning-critical components while preserving cross-modal alignment and accounting for layer-wise sensitivity under a global parameter budget. Experiments on four VLMs across three reasoning benchmarks show that MuCRASP consistently preserves reasoning quality under increasing compression. At 30% pruning on Qwen2.5-VL-7B, MuCRASP achieves an LLM-as-a-Judge score of 8.87 versus 7.32 for the strongest baseline on physical reasoning tasks. Furthermore, MuCRASP maintains high reasoning consistency up to 50% pruning, significantly outperforming prior pruning approaches while exhibiting lower perplexity degradation.
