Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
Abstract
Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several multi-level structures to learn a VAE model to generate long, and coherent text. In particular, we use a hierarchy of stochastic layers between the encoder and decoder networks to generate more informative latent codes. We also investigate a multi-level decoder structure to learn a coherent long-term structure by generating intermediate sentence representations as high-level plan vectors. Empirical results demonstrate that a multi-level VAE model produces more coherent and less repetitive long text compared to the standard VAE models and can further mitigate the posterior-collapse issue.
Cite
@article{arxiv.1902.00154,
title = {Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models},
author = {Dinghan Shen and Asli Celikyilmaz and Yizhe Zhang and Liqun Chen and Xin Wang and Jianfeng Gao and Lawrence Carin},
journal= {arXiv preprint arXiv:1902.00154},
year = {2019}
}
Comments
To appear at ACL 2019