In this paper, we investigate the tendency of end-to-end neural Machine Reading Comprehension (MRC) models to match shallow patterns rather than perform inference-oriented reasoning on RC benchmarks. We aim to test the ability of these systems to answer questions which focus on referential inference. We propose ParallelQA, a strategy to formulate such questions using parallel passages. We also demonstrate that existing neural models fail to generalize well to this setting.
@article{arxiv.1805.03830,
title = {Towards Inference-Oriented Reading Comprehension: ParallelQA},
author = {Soumya Wadhwa and Varsha Embar and Matthias Grabmair and Eric Nyberg},
journal= {arXiv preprint arXiv:1805.03830},
year = {2018}
}
Comments
Accepted at Workshop on New Forms of Generalization in Deep Learning and Natural Language Processing, NAACL 2018