Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
Computation and Language
2021-11-03 v1 Artificial Intelligence
Machine Learning
Abstract
Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field. We present a survey of recent work that uses these large language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches. We also present approaches that use pre-trained language models to generate data for training augmentation or other purposes. We conclude with discussions on limitations and suggested directions for future research.
Cite
@article{arxiv.2111.01243,
title = {Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey},
author = {Bonan Min and Hayley Ross and Elior Sulem and Amir Pouran Ben Veyseh and Thien Huu Nguyen and Oscar Sainz and Eneko Agirre and Ilana Heinz and Dan Roth},
journal= {arXiv preprint arXiv:2111.01243},
year = {2021}
}