Comments of online articles provide extended views and improve user engagement. Automatically making comments thus become a valuable functionality for online forums, intelligent chatbots, etc. This paper proposes the new task of automatic article commenting, and introduces a large-scale Chinese dataset with millions of real comments and a human-annotated subset characterizing the comments' varying quality. Incorporating the human bias of comment quality, we further develop automatic metrics that generalize a broad set of popular reference-based metrics and exhibit greatly improved correlations with human evaluations.
@article{arxiv.1805.03668,
title = {Automatic Article Commenting: the Task and Dataset},
author = {Lianhui Qin and Lemao Liu and Victoria Bi and Yan Wang and Xiaojiang Liu and Zhiting Hu and Hai Zhao and Shuming Shi},
journal= {arXiv preprint arXiv:1805.03668},
year = {2018}
}
Comments
ACL2018; with supplements; Dataset link available in the paper