English

Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation

Computer Vision and Pattern Recognition 2026-07-31 v1

Abstract

Multi-modal object Re-Identification (ReID) aims to retrieve target instances by leveraging complementary information across modalities. However, existing methods suffer from two challenges. First, they often fail to exploit well-aligned and reliable semantic priors, making them vulnerable to background clutter and cross-modal misalignment. On the other hand, they typically rely on holistic feature modeling, overlooking the synergy between global and local representations. To overcome these limitations, we propose a robust multi-modal ReID framework with dual semantic guidance and global-local mutual modulation, which mainly consists of three key components, namely the Text-Semantic Injector (TSI), the Masked Global-Local Modulator (MGLM), and the Hierarchical MoE Fusion (HMF). The TSI enhances semantic awareness by integrating clean and coherent textual features into visual tokens. The MGLM enables part-aware cross-modal interaction through joint guidance from soft masks and global context, improving fine-grained feature alignment. Finally, the HMF adaptively aggregates multi-spectral features under local semantic supervision, yielding discriminative and robust representations. Extensive experiments on three multi-modal ReID benchmarks demonstrate the effectiveness of the proposed method. The code will be made publicly available at https://github.com/zw-absin/DSGM upon acceptance.

Cite

@article{arxiv.2607.29207,
  title  = {Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation},
  author = {Weixiang Zhou and Xingguo Xu and Yuhao Wang and Cong Wang and Yang Yang and Zhixun Su and Jinshan Pan},
  journal= {arXiv preprint arXiv:2607.29207},
  year   = {2026}
}

Comments

Accepted by IEEE TCSVT 2026. The version of record may differ slightly