This paper introduces the SAMSum Corpus, a new dataset with abstractive dialogue summaries. We investigate the challenges it poses for automated summarization by testing several models and comparing their results with those obtained on a corpus of news articles. We show that model-generated summaries of dialogues achieve higher ROUGE scores than the model-generated summaries of news -- in contrast with human evaluators' judgement. This suggests that a challenging task of abstractive dialogue summarization requires dedicated models and non-standard quality measures. To our knowledge, our study is the first attempt to introduce a high-quality chat-dialogues corpus, manually annotated with abstractive summarizations, which can be used by the research community for further studies.
@article{arxiv.1911.12237,
title = {SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization},
author = {Bogdan Gliwa and Iwona Mochol and Maciej Biesek and Aleksander Wawer},
journal= {arXiv preprint arXiv:1911.12237},
year = {2019}
}
Comments
Attachment contains the described dataset archived in 7z format. Please see the attached readme and licence. Update of the previous version: changed formats of train/val/test files in corpus.7z