Shape of Elephant: Study of Macro Properties of Word Embeddings Spaces
Computation and Language
2021-06-15 v1 Machine Learning
Abstract
Pre-trained word representations became a key component in many NLP tasks. However, the global geometry of the word embeddings remains poorly understood. In this paper, we demonstrate that a typical word embeddings cloud is shaped as a high-dimensional simplex with interpretable vertices and propose a simple yet effective method for enumeration of these vertices. We show that the proposed method can detect and describe vertices of the simplex for GloVe and fasttext spaces.
Cite
@article{arxiv.2106.06964,
title = {Shape of Elephant: Study of Macro Properties of Word Embeddings Spaces},
author = {Alexey Tikhonov},
journal= {arXiv preprint arXiv:2106.06964},
year = {2021}
}
Comments
3 pages, 2 figures, EEML-2021 poster