We present an approach named the Cycled Composition Network that can measure the semantic distance of the composition of image-text embedding. First, the Composition Network transit a reference image to target image in an embedding space using relative caption. Second, the Correction Network calculates a difference between reference and retrieved target images in the embedding space and match it with a relative caption. Our goal is to learn a Composition mapping with the Composition Network. Since this one-way mapping is highly under-constrained, we couple it with an inverse relation learning with the Correction Network and introduce a cycled relation for given Image We participate in Fashion IQ 2020 challenge and have won the first place with the ensemble of our model.
@article{arxiv.2107.11509,
title = {Cycled Compositional Learning between Images and Text},
author = {Jongseok Kim and Youngjae Yu and Seunghwan Lee and GunheeKim},
journal= {arXiv preprint arXiv:2107.11509},
year = {2021}
}
Comments
Fashion IQ 2020 challenge winner. Workshop tech report