An Essay on Optimization Mystery of Deep Learning
Machine Learning
2019-05-20 v1 Machine Learning
Abstract
Despite the huge empirical success of deep learning, theoretical understanding of neural networks learning process is still lacking. This is the reason, why some of its features seem "mysterious". We emphasize two mysteries of deep learning: generalization mystery, and optimization mystery. In this essay we review and draw connections between several selected works concerning the latter.
Keywords
Cite
@article{arxiv.1905.07187,
title = {An Essay on Optimization Mystery of Deep Learning},
author = {Eugene Golikov},
journal= {arXiv preprint arXiv:1905.07187},
year = {2019}
}