Sequentially Controlled Text Generation
Computation and Language
2023-01-09 v1 Artificial Intelligence
Machine Learning
Abstract
While GPT-2 generates sentences that are remarkably human-like, longer documents can ramble and do not follow human-like writing structure. We study the problem of imposing structure on long-range text. We propose a novel controlled text generation task, sequentially controlled text generation, and identify a dataset, NewsDiscourse as a starting point for this task. We develop a sequential controlled text generation pipeline with generation and editing. We test different degrees of structural awareness and show that, in general, more structural awareness results in higher control-accuracy, grammaticality, coherency and topicality, approaching human-level writing performance.
Cite
@article{arxiv.2301.02299,
title = {Sequentially Controlled Text Generation},
author = {Alexander Spangher and Xinyu Hua and Yao Ming and Nanyun Peng},
journal= {arXiv preprint arXiv:2301.02299},
year = {2023}
}
Comments
19 pages. 10 pages main body, 3 pages references, 6 pages appendix