Linear discriminant initialization for feed-forward neural networks
Machine Learning
2020-08-19 v2 Metric Geometry
Machine Learning
Abstract
Informed by the basic geometry underlying feed forward neural networks, we initialize the weights of the first layer of a neural network using the linear discriminants which best distinguish individual classes. Networks initialized in this way take fewer training steps to reach the same level of training, and asymptotically have higher accuracy on training data.
Cite
@article{arxiv.2007.12782,
title = {Linear discriminant initialization for feed-forward neural networks},
author = {Marissa Masden and Dev Sinha},
journal= {arXiv preprint arXiv:2007.12782},
year = {2020}
}
Comments
6 pages, 7 figures. Added comparison to larger randomly initialized networks. Updated tables to better illustrate trends in effect size