A note on sample complexity of learning binary output neural networks under fixed input distributions
Machine Learning
2016-11-17 v1
Abstract
We show that the learning sample complexity of a sigmoidal neural network constructed by Sontag (1992) required to achieve a given misclassification error under a fixed purely atomic distribution can grow arbitrarily fast: for any prescribed rate of growth there is an input distribution having this rate as the sample complexity, and the bound is asymptotically tight. The rate can be superexponential, a non-recursive function, etc. We further observe that Sontag's ANN is not Glivenko-Cantelli under any input distribution having a non-atomic part.
Cite
@article{arxiv.1007.1282,
title = {A note on sample complexity of learning binary output neural networks under fixed input distributions},
author = {Vladimir Pestov},
journal= {arXiv preprint arXiv:1007.1282},
year = {2016}
}
Comments
6 pages, latex in IEEE conference proceedings format