English

Zero-Shot Compositional Concept Learning

Computer Vision and Pattern Recognition 2021-07-13 v1 Computation and Language

Abstract

In this paper, we study the problem of recognizing compositional attribute-object concepts within the zero-shot learning (ZSL) framework. We propose an episode-based cross-attention (EpiCA) network which combines merits of cross-attention mechanism and episode-based training strategy to recognize novel compositional concepts. Firstly, EpiCA bases on cross-attention to correlate concept-visual information and utilizes the gated pooling layer to build contextualized representations for both images and concepts. The updated representations are used for a more in-depth multi-modal relevance calculation for concept recognition. Secondly, a two-phase episode training strategy, especially the transductive phase, is adopted to utilize unlabeled test examples to alleviate the low-resource learning problem. Experiments on two widely-used zero-shot compositional learning (ZSCL) benchmarks have demonstrated the effectiveness of the model compared with recent approaches on both conventional and generalized ZSCL settings.

Keywords

Cite

@article{arxiv.2107.05176,
  title  = {Zero-Shot Compositional Concept Learning},
  author = {Guangyue Xu and Parisa Kordjamshidi and Joyce Y. Chai},
  journal= {arXiv preprint arXiv:2107.05176},
  year   = {2021}
}