Do Explanations make VQA Models more Predictable to a Human?
Artificial Intelligence
2018-10-31 v1 Computation and Language
Computer Vision and Pattern Recognition
Abstract
A rich line of research attempts to make deep neural networks more transparent by generating human-interpretable 'explanations' of their decision process, especially for interactive tasks like Visual Question Answering (VQA). In this work, we analyze if existing explanations indeed make a VQA model -- its responses as well as failures -- more predictable to a human. Surprisingly, we find that they do not. On the other hand, we find that human-in-the-loop approaches that treat the model as a black-box do.
Cite
@article{arxiv.1810.12366,
title = {Do Explanations make VQA Models more Predictable to a Human?},
author = {Arjun Chandrasekaran and Viraj Prabhu and Deshraj Yadav and Prithvijit Chattopadhyay and Devi Parikh},
journal= {arXiv preprint arXiv:1810.12366},
year = {2018}
}
Comments
EMNLP 2018. 16 pages, 11 figures. Content overlaps with "It Takes Two to Tango: Towards Theory of AI's Mind" (arXiv:1704.00717)