GLOSS: Generative Latent Optimization of Sentence Representations
Computation and Language
2019-08-14 v1 Machine Learning
Abstract
We propose a method to learn unsupervised sentence representations in a non-compositional manner based on Generative Latent Optimization. Our approach does not impose any assumptions on how words are to be combined into a sentence representation. We discuss a simple Bag of Words model as well as a variant that models word positions. Both are trained to reconstruct the sentence based on a latent code and our model can be used to generate text. Experiments show large improvements over the related Paragraph Vectors. Compared to uSIF, we achieve a relative improvement of 5% when trained on the same data and our method performs competitively to Sent2vec while trained on 30 times less data.
Cite
@article{arxiv.1907.06385,
title = {GLOSS: Generative Latent Optimization of Sentence Representations},
author = {Sidak Pal Singh and Angela Fan and Michael Auli},
journal= {arXiv preprint arXiv:1907.06385},
year = {2019}
}