English

I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking

Computer Vision and Pattern Recognition 2025-08-05 v1 Information Retrieval

Abstract

Multimodal entity linking plays a crucial role in a wide range of applications. Recent advances in large language model-based methods have become the dominant paradigm for this task, effectively leveraging both textual and visual modalities to enhance performance. Despite their success, these methods still face two challenges, including unnecessary incorporation of image data in certain scenarios and the reliance only on a one-time extraction of visual features, which can undermine their effectiveness and accuracy. To address these challenges, we propose a novel LLM-based framework for the multimodal entity linking task, called Intra- and Inter-modal Collaborative Reflections. This framework prioritizes leveraging text information to address the task. When text alone is insufficient to link the correct entity through intra- and inter-modality evaluations, it employs a multi-round iterative strategy that integrates key visual clues from various aspects of the image to support reasoning and enhance matching accuracy. Extensive experiments on three widely used public datasets demonstrate that our framework consistently outperforms current state-of-the-art methods in the task, achieving improvements of 3.2%, 5.1%, and 1.6%, respectively. Our code is available at https://github.com/ziyan-xiaoyu/I2CR/.

Keywords

Cite

@article{arxiv.2508.02243,
  title  = {I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking},
  author = {Ziyan Liu and Junwen Li and Kaiwen Li and Tong Ruan and Chao Wang and Xinyan He and Zongyu Wang and Xuezhi Cao and Jingping Liu},
  journal= {arXiv preprint arXiv:2508.02243},
  year   = {2025}
}

Comments

10 pages, 6 figures, accepted by ACMMM 2025

R2 v1 2026-07-01T04:32:59.354Z