Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting ExNLP datasets in the future.
@article{arxiv.2102.12060,
title = {Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing},
author = {Sarah Wiegreffe and Ana Marasović},
journal= {arXiv preprint arXiv:2102.12060},
year = {2021}
}
Comments
v3: NeurIPS 2021 accepted paper camera-ready version. The content of v3 is almost the same as of v1-2 but is more condensed. v4: Fixed a typo in the title and added acknowledgements. 10 pages main, 6 pages appendix