Word2vec Conjecture and A Limitative Result
Abstract
Being inspired by the success of \texttt{word2vec} \citep{mikolov2013distributed} in capturing analogies, we study the conjecture that analogical relations can be represented by vector spaces. Unlike many previous works that focus on the distributional semantic aspect of \texttt{word2vec}, we study the purely \emph{representational} question: can \emph{all} semantic word-word relations be represented by differences (or directions) of vectors? We call this the word2vec conjecture and point out some of its desirable implications. However, we will exhibit a class of relations that cannot be represented in this way, thus falsifying the conjecture and establishing a limitative result for the representability of semantic relations by vector spaces over fields of characteristic 0, e.g., real or complex numbers.
Cite
@article{arxiv.2010.12719,
title = {Word2vec Conjecture and A Limitative Result},
author = {Falcon Z. Dai},
journal= {arXiv preprint arXiv:2010.12719},
year = {2020}
}