Size-Independent Sample Complexity of Neural Networks
Machine Learning
2019-11-19 v5 Neural and Evolutionary Computing
Machine Learning
Abstract
We study the sample complexity of learning neural networks, by providing new bounds on their Rademacher complexity assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth, and under some additional assumptions, are fully independent of the network size (both depth and width). These results are derived using some novel techniques, which may be of independent interest.
Keywords
Cite
@article{arxiv.1712.06541,
title = {Size-Independent Sample Complexity of Neural Networks},
author = {Noah Golowich and Alexander Rakhlin and Ohad Shamir},
journal= {arXiv preprint arXiv:1712.06541},
year = {2019}
}
Comments
Fixed a bug in the proof of theorem 7 (not affecting theorem statement), by slightly changing the construction