Multimodal Word Distributions
Machine Learning
2019-09-10 v2 Artificial Intelligence
Computation and Language
Machine Learning
Abstract
Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective. We show that the resulting approach captures uniquely expressive semantic information, and outperforms alternatives, such as word2vec skip-grams, and Gaussian embeddings, on benchmark datasets such as word similarity and entailment.
Cite
@article{arxiv.1704.08424,
title = {Multimodal Word Distributions},
author = {Ben Athiwaratkun and Andrew Gordon Wilson},
journal= {arXiv preprint arXiv:1704.08424},
year = {2019}
}
Comments
This paper also appears at ACL 2017