There is an increasing interest in continuous learning (CL), as data privacy is becoming a priority for real-world machine learning applications. Meanwhile, there is still a lack of academic NLP benchmarks that are applicable for realistic CL settings, which is a major challenge for the advancement of the field. In this paper we discuss some of the unrealistic data characteristics of public datasets, study the challenges of realistic single-task continuous learning as well as the effectiveness of data rehearsal as a way to mitigate accuracy loss. We construct a CL NER dataset from an existing publicly available dataset and release it along with the code to the research community.
@article{arxiv.2110.14694,
title = {Towards Realistic Single-Task Continuous Learning Research for NER},
author = {Justin Payan and Yuval Merhav and He Xie and Satyapriya Krishna and Anil Ramakrishna and Mukund Sridhar and Rahul Gupta},
journal= {arXiv preprint arXiv:2110.14694},
year = {2021}
}
Comments
11 pages, 2 figures, Findings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) (short paper), November 2021