English

MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning

Artificial Intelligence 2026-05-26 v1 Computation and Language

Abstract

Vision-language models (VLMs) increasingly rely on chain-of-thought (CoT) reasoning to solve complex multimodal tasks, but their large parameter sizes make deployment expensive. Structured pruning offers a natural solution; however, existing methods fail to preserve CoT reasoning accuracy in VLMs. We identify two key reasons: (1) CoT consistency depends on sparse transition points (pivot tokens) in the generation trajectory, while existing pruning methods are CoT-agnostic; and (2) pruning methods designed for unimodal LLMs do not account for activation-distribution differences across visual and textual modalities. Motivated by these observations, we propose MuCRASP, a structured pruning framework that targets reasoning-critical components while preserving cross-modal alignment and accounting for layer-wise sensitivity under a global parameter budget. Experiments on four VLMs across three reasoning benchmarks show that MuCRASP consistently preserves reasoning quality under increasing compression. At 30% pruning on Qwen2.5-VL-7B, MuCRASP achieves an LLM-as-a-Judge score of 8.87 versus 7.32 for the strongest baseline on physical reasoning tasks. Furthermore, MuCRASP maintains high reasoning consistency up to 50% pruning, significantly outperforming prior pruning approaches while exhibiting lower perplexity degradation.

Keywords

Cite

@article{arxiv.2605.25842,
  title  = {MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning},
  author = {Aritra Dutta and Somak Aditya},
  journal= {arXiv preprint arXiv:2605.25842},
  year   = {2026}
}

Comments

First Preprint

R2 v1 2026-07-22T07:32:31.774Z