word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA
Computation and Language
2017-05-30 v1 Machine Learning
Abstract
We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.
Cite
@article{arxiv.1705.09755,
title = {word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA},
author = {Andrew J. Landgraf and Jeremy Bellay},
journal= {arXiv preprint arXiv:1705.09755},
year = {2017}
}