English

ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval

Computer Vision and Pattern Recognition 2026-04-01 v2

Abstract

Composed Image Retrieval (CIR) aims to retrieve target images based on a hybrid query comprising a reference image and a modification text. Early dual-tower Vision-Language Models (VLMs) struggle with cross-modality compositional reasoning required for this task. While adapting generative Multimodal Large Language Models (MLLMs) for retrieval offers a promising direction, we identify that this strategy overlooks a fundamental issue: compressing a generative MLLM into a single-embedding discriminative retriever triggers a paradigm conflict, which leads to Capability Degradation - the deterioration of native fine-grained reasoning after retrieval adaptation. To address this challenge, we propose ReCALL, a model-agnostic framework that follows a diagnose-generate-refine pipeline: First, we diagnose cognitive blind spots of the retriever via self-guided informative instance mining. Next, we generate corrective instructions and triplets by prompting the foundation MLLM and conduct quality control with VQA-based consistency filtering. Finally, we refine the retriever through continual training on these triplets with a grouped contrastive scheme, thereby internalizing fine-grained visual-semantic distinctions and realigning the discriminative embedding space of retriever with intrinsic compositional reasoning within the MLLM. Extensive experiments on CIRR and FashionIQ show that ReCALL consistently recalibrates degraded capabilities and achieves state-of-the-art performance. Code is available at https://github.com/RemRico/Recall.

Keywords

Cite

@article{arxiv.2602.01639,
  title  = {ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval},
  author = {Tianyu Yang and Chenwei He and Xiangzhao Hao and Tianyue Wang and Jiarui Guo and Haiyun Guo and Leigang Qu and Jinqiao Wang and Tat-Seng Chua},
  journal= {arXiv preprint arXiv:2602.01639},
  year   = {2026}
}

Comments

Accepted to CVPR 2026

R2 v1 2026-07-01T09:30:55.540Z