We introduce a new inference task - Visual Entailment (VE) - which differs from traditional Textual Entailment (TE) tasks whereby a premise is defined by an image, rather than a natural language sentence as in TE tasks. A novel dataset SNLI-VE (publicly available at https://github.com/necla-ml/SNLI-VE) is proposed for VE tasks based on the Stanford Natural Language Inference corpus and Flickr30k. We introduce a differentiable architecture called the Explainable Visual Entailment model (EVE) to tackle the VE problem. EVE and several other state-of-the-art visual question answering (VQA) based models are evaluated on the SNLI-VE dataset, facilitating grounded language understanding and providing insights on how modern VQA based models perform.
@article{arxiv.1811.10582,
title = {Visual Entailment Task for Visually-Grounded Language Learning},
author = {Ning Xie and Farley Lai and Derek Doran and Asim Kadav},
journal= {arXiv preprint arXiv:1811.10582},
year = {2019}
}
Comments
4 pages, accepted by Visually Grounded Interaction and Language (ViGIL) workshop in NeurIPS 2018