This paper aims for the language-based product image retrieval task. The majority of previous works have made significant progress by designing network structure, similarity measurement, and loss function. However, they typically perform vision-text matching at certain granularity regardless of the intrinsic multiple granularities of images. In this paper, we focus on the cross-modal similarity measurement, and propose a novel Hierarchical Similarity Learning (HSL) network. HSL first learns multi-level representations of input data by stacked encoders, and object-granularity similarity and image-granularity similarity are computed at each level. All the similarities are combined as the final hierarchical cross-modal similarity. Experiments on a large-scale product retrieval dataset demonstrate the effectiveness of our proposed method. Code and data are available at https://github.com/liufh1/hsl.
@article{arxiv.2102.09375,
title = {Hierarchical Similarity Learning for Language-based Product Image Retrieval},
author = {Zhe Ma and Fenghao Liu and Jianfeng Dong and Xiaoye Qu and Yuan He and Shouling Ji},
journal= {arXiv preprint arXiv:2102.09375},
year = {2021}
}
Comments
Accepted by ICASSP 2021. Code and data will be available at https://github.com/liufh1/hsl